Trying to understand UVC Part 1

This will be a series of blogs, where I go about trying to reverse engineer(but not really reverse engineer, more like track the raw data) UVC using linux. The goal is to try to hack into the code starting from guvcview and track where the raw data is coming from as far as possible. Perhaps I will hit a dead end, but I hope I learn something also.

Full disclosure: I don’t know much about C at this point, just have a few tricks up my sleeves, so I may make incorrect statements, but I need as a way to log my work and hope that this will help other people also.

I downloaded the repo from: https://sourceforge.net/p/guvcview/git-master/ci/master/tree/

Here is what I cloned:

git clone git://git.code.sf.net/p/guvcview/git-master guvcview

I am using Ubuntu 18.04 on celeron processor.

I started by installing everything using the information in the README file.

I also downloaded v4l-utils folder

this is how I make the v4l-utils:

git clone git://linuxtv.org/v4l-utils.git

cd ~/v4l-utils/;sudo make;sudo make install

I use the 2nd line when I am making v4l-utils

Then I opened up the binary to see if it would by using the following command:

p0xxx:~/git-repos/guvcview$ ./guvcview/guvcview –help
Guvcview version 2.0.6

Usage:
guvcview [OPTIONS]

OPTIONS:
-h,–help :Print help
-v,–version :Print version
-w,–verbosity=LEVEL :Set Verbosity level (def: 0)
-d,–device=DEVICE :Set device name (def: /dev/video0)
-c,–capture=METHOD :Set capture method [read | mmap (def)]
-b,–disable_libv4l2 :disable calls to libv4l2
-x,–resolution=WIDTHxHEIGHT :Request resolution (e.g 640×480)
-f,–format=FOURCC :Request format (e.g MJPG)
-F,–fps=FPS_NUM[/FPS_DENOM] :Request fps (e.g 25 ; 7/3; …)
-r,–render=RENDER_API :Select render API (e.g none; sdl; sfml)
-m,–render_window=RENDER_WINDOW_FLAGS :Set render window flags (e.g none; full; max; WIDTHxHEIGHT)
-a,–audio=AUDIO_API :Select audio API (e.g none; port; pulse)
-k,–audio_device=AUDIO_DEVICE :Select audio device index for selected api (0..N)
-g,–gui=GUI_API :Select GUI API (e.g none)
-o,–audio_codec=CODEC :Audio codec [pcm mp2 mp3 aac ac3 vorb]
-u,–video_codec=CODEC :Video codec [raw mjpg mpeg flv1 wmv1 mpg2 mp43 dx50 h264 vp80 theo]
-p,–profile=FILENAME :load control profile
-j,–video=FILENAME :filename for captured video)
-i,–image=FILENAME :filename for captured image)
-y,–video_timer=TIME_IN_SEC :time (double) in sec. for video capture)
-t,–photo_timer=TIME_IN_SEC :time (double) in sec. between captured photos)
-n,–photo_total=TOTAL :total number of captured photos)
-e,–exit_on_term :exit app after video or image capture ends
-z,–control_panel :Start in control panel mode

There are a lot of things to do. The command I am using right now3 is:

xx:~/git-repos/guvcview$ ./guvcview/guvcview –verbosity=10 –render_window=FULL –video_code=raw -d /dev/video0 –format=yuyv

The first thing I did was to turn on verbosity, see if it can provide any clues as to how I should start hacking.

0-p0xxx:~/git-repos/guvcview$ ~/git-repos/guvcview/guvcview/guvcview –verbosity=10 –render_window=FULL –video_code=raw -d /dev/video0 –format=yuyv
GUVCVIEW: version 2.0.6
GUVCVIEW: language catalog=> dir:/usr/local/share/locale type:en_US.UTF-8 cat:guvcview.mo
Guvcview: using GUI 0 for option non
GUVCVIEW: main thread (tid: 23733)
V4L2_CORE: language catalog=> dir:/usr/local/share/locale type:en_US.UTF-8 cat:gview_v4l2core.mo
V4L2_CORE: capture method mmap (1)
V4L2_CORE: video device: /dev/video0
V4L2_CORE: mapping control for Pan (relative)
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for Tilt (relative)
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for Pan Reset
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for Tilt Reset
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for Focus
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for LED1 Mode
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for LED1 Frequency
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for Disable video processing
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: mapping control for Raw bits per pixel
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: Init. HD Pro Webcam C920 (location: usb-0000:00:14.0-3)
{ pixelformat = ‘YUYV’, description = ‘YUYV 4:2:2’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/5,
{ discrete: width = 2304, height = 1296 }
Time interval between frame: 1/2,
{ discrete: width = 2304, height = 1536 }
Time interval between frame: 1/2,
{ pixelformat = ‘H264’, description = ‘H.264’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ pixelformat = ‘MJPG’, description = ‘Motion-JPEG’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ pixelformat = ‘RGB3’, description = ‘RGB3’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 2304, height = 1296 }
Time interval between frame: 1/2,
{ discrete: width = 2304, height = 1536 }
Time interval between frame: 1/2,
{ pixelformat = ‘BGR3’, description = ‘BGR3’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 2304, height = 1296 }
Time interval between frame: 1/2,
{ discrete: width = 2304, height = 1536 }
Time interval between frame: 1/2,
{ pixelformat = ‘YU12’, description = ‘YU12’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 2304, height = 1296 }
Time interval between frame: 1/2,
{ discrete: width = 2304, height = 1536 }
Time interval between frame: 1/2,
{ pixelformat = ‘YV12’, description = ‘YV12’ }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 90 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 160, height = 120 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 176, height = 144 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 180 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 320, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 432, height = 240 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 640, height = 360 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 448 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 800, height = 600 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 864, height = 480 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 960, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1024, height = 576 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1280, height = 720 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1600, height = 896 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 1920, height = 1080 }
Time interval between frame: 1/30, 1/24, 1/20, 1/15, 1/10, 2/15, 1/5,
{ discrete: width = 2304, height = 1296 }
Time interval between frame: 1/2,
{ discrete: width = 2304, height = 1536 }
Time interval between frame: 1/2,
V4L2_CORE: checking muxed H264 format support
V4L2_CORE: H264 format already in list
V4L2_CORE: checking for UVCX_H264 unit id
V4L2_CORE: checking pan/tilt unit id for device 0 (bus:1 dev:9)
V4L2_CORE: (libusb) checking bus(2) dev(1) for device
V4L2_CORE: (libusb) checking bus(1) dev(6) for device
V4L2_CORE: (libusb) checking bus(1) dev(5) for device
V4L2_CORE: (libusb) checking bus(1) dev(4) for device
V4L2_CORE: (libusb) checking bus(1) dev(9) for device
V4L2_CORE: (libusb) checking for GUID unit id
V4L2_CORE: (libusb) found GUID unit id 12
control[0]:(int) 0x980900 ‘Brightness’
min:0 max:255 step:1 def:128 curr:0
control[1]:(int) 0x980901 ‘Contrast’
min:0 max:255 step:1 def:128 curr:0
control[2]:(int) 0x980902 ‘Saturation’
min:0 max:255 step:1 def:128 curr:0
control[3]:(bool) 0x98090c ‘White Balance Temperature, Auto’
def:1 curr:0
control[4]:(int) 0x980913 ‘Gain’
min:0 max:255 step:1 def:0 curr:0
control[5]:(menu) 0x980918 ‘Power Line Frequency’
min:0 max:2 def:2 curr:0
menu[0]: [0] -> ‘Disabled’
menu[1]: [1] -> ’50 Hz’
menu[2]: [2] -> ’60 Hz’
control[6]:(int) 0x98091a ‘White Balance Temperature’
min:2000 max:6500 step:1 def:4000 curr:0
control[7]:(int) 0x98091b ‘Sharpness’
min:0 max:255 step:1 def:128 curr:0
control[8]:(int) 0x98091c ‘Backlight Compensation’
min:0 max:1 step:1 def:0 curr:0
control[9]:(menu) 0x9a0901 ‘Exposure, Auto’
min:0 max:4 def:3 curr:0
menu[0]: [1] -> ‘Manual Mode’
menu[1]: [3] -> ‘Aperture Priority Mode’
menu[2]: [4] -> ‘Fuck you fucking fuck’
control[10]:(int) 0x9a0902 ‘Exposure (Absolute)’
min:3 max:2047 step:1 def:250 curr:0
control[11]:(bool) 0x9a0903 ‘Exposure, Auto Priority’
def:0 curr:0
control[12]:(int) 0x9a0908 ‘Pan (Absolute)’
min:-36000 max:36000 step:3600 def:0 curr:0
control[13]:(int) 0x9a0909 ‘Tilt (Absolute)’
min:-36000 max:36000 step:3600 def:0 curr:0
control[14]:(int) 0x9a090a ‘Focus (absolute)’
min:0 max:250 step:5 def:0 curr:0
control[15]:(bool) 0x9a090c ‘Focus, Auto’
def:1 curr:0
control[16]:(int) 0x9a090d ‘Zoom, Absolute’
min:100 max:500 step:1 def:100 curr:0
control[17]:(menu) 0xa046d05 ‘LED1 Mode’
min:0 max:3 def:0 curr:0
menu[0]: [0] -> ‘Off’
menu[1]: [1] -> ‘On’
menu[2]: [2] -> ‘Blink’
menu[3]: [3] -> ‘Auto’
control[18]:(int) 0xa046d06 ‘LED1 Frequency’
min:0 max:255 step:1 def:0 curr:0
V4L2_CORE: fps configured to 1/25
GUVCVIEW: setting video codec to ‘raw’
GUVCVIEW: setting audio codec to ‘mp2’
ENCODER: audio codec list size:6
ENCODER: audio codec list size:6
ENCODER: audio codec list size:6
ENCODER: audio codec list size:6
ENCODER: audio codec list size:6
GUVCVIEW: basename for my_video.mkv is my_video.mkv
GUVCVIEW: extension for my_video.mkv is mkv
GUVCVIEW: basename for my_photo.jpg is my_photo.jpg
GUVCVIEW: extension for my_photo.jpg is jpg
ALSA lib pcm_dmix.c:1052:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_dmix.c:1052:(snd_pcm_dmix_open) unable to open slave

This led me to a clue that I was looking for:

RENDER: Initializing SDL2 render
RENDER: video display 0 -> 1920x1080px @ 60hz
RENDER: setting window size to 640×480

RENDER: Available SDL2 rendering drivers:
0: opengl
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [ ]
SDL_RENDERER_ACCELERATED [X]
SDL_RENDERER_PRESENTVSYNC [X]
1: opengles2
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [ ]
SDL_RENDERER_ACCELERATED [X]
SDL_RENDERER_PRESENTVSYNC [X]
2: software
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [X]
SDL_RENDERER_ACCELERATED [ ]
SDL_RENDERER_PRESENTVSYNC [ ]
RENDER: rendering driver in use: opengl
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [ ]
SDL_RENDERER_ACCELERATED [X]
SDL_RENDERER_PRESENTVSYNC [X]
V4L2_CORE: (VIDIOC_STREAMON) stream_status = STRM_OK

I decided to find where SDL_RENDERER is used. I looked for files that had “sdl” in them with a find command:

Here was what I saw:

p0xxx:~/git-repos/guvcview$ find . -name “*sdl*”
./gview_render/render_sdl2.h
./gview_render/libgviewrender_la-render_sdl2.lo
./gview_render/render_sdl2.c
./gview_render/.deps/libgviewrender_la-render_sdl2.Plo
./gview_render/libgviewrender_la-render_sdl2.o
./gview_render/.libs/libgviewrender_la-render_sdl2.o

I figured that the .h file was not useful so I wanted to go into the .c because everything else I didn’t know what to do with.

I always take a picture before and after I make a change to see its impact. Right before I made a change my image looked like below:

Screenshot from 2019-05-09 08-11-25

Then I searched for render and found a function that sounded promising:

I compiled and ran the program with following commands:

cd ~/git-repos/guvcview/;./configure && make && sudo make install

~/git-repos/guvcview/guvcview/guvcview –verbosity=10 –render_window=FULL –video_code=raw -d /dev/video0 –format=yuyv

The reason to go with raw and yuv is because I wanted to remove compression.

I modified the following function and added the items in bold and italics:

int render_sdl2_frame(uint8_t *frame, int width, int height)
{
/*asserts*/
assert(rending_texture != NULL);
assert(frame != NULL);
int i =0;
uint8_t *tmp=frame;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}

SDL_SetRenderDrawColor(main_renderer, 0, 0, 0, 255); /*black*/
SDL_RenderClear(main_renderer);

/* since data is continuous we can use SDL_UpdateTexture
* instead of SDL_UpdateYUVTexture.
* no need to use SDL_Lock/UnlockTexture (it doesn’t seem faster)
*/
SDL_UpdateTexture(rending_texture, NULL, frame, width);

SDL_RenderCopy(main_renderer, rending_texture, NULL, NULL);

SDL_RenderPresent(main_renderer);

return 0;
}

The goal was if I could impact the image by setting the first couple “pixels” to zeros. Since this is a yuv I don’t know how much impact it would have but I figured it should be noticeable. Here is what the image looked like after implementing the for loop:

Screenshot from 2019-05-09 08-17-08

If you look compare the before and after image you will see the top of the image has been removed. Success! This means I can do something!

I then looked for where the function was called with the following grep command:

0xxx:~/git-repos/guvcview$ grep -r –include “*.c” render_sdl2_frame
gview_render/render_sdl2.c:int render_sdl2_frame(uint8_t *frame, int width, int height)
gview_render/render.c: ret = render_sdl2_frame(frame, my_width, my_height);

There is another file called render.c so I decided to checkout where that function is called. When i searched the render.c file I found the following function:

/*
* render a frame
* args:
* frame – pointer to frame data (yu12 format)
*
* asserts:
* frame is not null
*
* returns: error code
*/
int render_frame(uint8_t *frame)
{
/*asserts*/
assert(frame != NULL);

int ret = 0;
switch(render_api)
{
case RENDER_NONE:
break;

#if ENABLE_SFML
case RENDER_SFML:
ret = render_sfml_frame(frame, my_width, my_height);
render_sfml_dispatch_events();
break;
#endif

#if ENABLE_SDL2
case RENDER_SDL:
ret = render_sdl2_frame(frame, my_width, my_height);
render_sdl2_dispatch_events();
break;
#endif

default:
break;
}

return ret;
}

At this point I decided this was too similar to render_sdl2_frame so I wanted to know where this function was being called, i.e. where is the frame variable coming from?

0xxx:~/git-repos/guvcview$ grep -r –include “*.c” render_frame
gview_render/render.c:void render_frame_fx(uint8_t *frame, uint32_t mask)
gview_render/render.c:void render_frame_osd(uint8_t *frame)
gview_render/render.c:int render_frame(uint8_t *frame)
guvcview/video_capture.c: render_frame_fx(frame->yuv_frame, my_render_mask);
guvcview/video_capture.c: render_frame_osd(frame->yuv_frame);
guvcview/video_capture.c: render_frame(frame->yuv_frame);

As you can see here there are 2 files that show up. One is defining it and the other is using it. So I decided to look at video_capture.c since I have already looked at render.c

I saw the following function and tried to edit the function (edit in BOLD)

/*
* capture loop (should run in a separate thread)
* args:
* data – pointer to user data (options data)
*
* asserts:
* none
*
* returns: pointer to return code
*/
void *capture_loop(void *data)
{
__LOCK_MUTEX(&capture_mutex);
capture_loop_data_t *cl_data = (capture_loop_data_t *) data;
options_t *my_options = (options_t *) cl_data->options;
config_t *my_config = (config_t *) cl_data->config;

uint64_t my_last_photo_time = 0; /*timer count*/
int my_photo_npics = 0;/*no npics*/

/*reset quit flag*/
quit = 0;

if(debug_level > 1)
printf(“GUVCVIEW: capture thread (tid: %u)\n”,
(unsigned int) syscall (SYS_gettid));

int ret = 0;

int render_flags = 0;

if (strcasecmp(my_options->render_flag, “full”) == 0)
render_flags = 1;
else if (strcasecmp(my_options->render_flag, “max”) == 0)
render_flags = 2;

render_set_verbosity(debug_level);

render_set_crosshair_color(my_config->crosshair_color);

if(render_init(
render,
v4l2core_get_frame_width(my_vd),
v4l2core_get_frame_height(my_vd),
render_flags,
my_options->render_width,
my_options->render_height) < 0) render = RENDER_NONE; else { render_set_event_callback(EV_QUIT, &quit_callback, NULL); render_set_event_callback(EV_KEY_V, &key_V_callback, NULL); render_set_event_callback(EV_KEY_I, &key_I_callback, NULL); render_set_event_callback(EV_KEY_UP, &key_UP_callback, NULL); render_set_event_callback(EV_KEY_DOWN, &key_DOWN_callback, NULL); render_set_event_callback(EV_KEY_LEFT, &key_LEFT_callback, NULL); render_set_event_callback(EV_KEY_RIGHT, &key_RIGHT_callback, NULL); } /*add a video capture timer*/ if(my_options->video_timer > 0)
{
my_video_timer = NSEC_PER_SEC * my_options->video_timer;
my_video_begin_time = v4l2core_time_get_timestamp(); /*timer count*/
/*if are not saving video start it*/
if(!get_encoder_status())
start_encoder_thread();
}

/*add a photo capture timer*/
if(my_options->photo_timer > 0)
{
my_photo_timer = NSEC_PER_SEC * my_options->photo_timer;
my_last_photo_time = v4l2core_time_get_timestamp(my_vd); /*timer count*/
}

if(my_options->photo_npics > 0)
my_photo_npics = my_options->photo_npics;

v4l2core_start_stream(my_vd);

v4l2_frame_buff_t *frame = NULL; //pointer to frame buffer

__COND_SIGNAL(&capture_cond);
__UNLOCK_MUTEX(&capture_mutex);

while(!quit)
{
if(restart)
{
int current_width = v4l2core_get_frame_width(my_vd);
int current_height = v4l2core_get_frame_height(my_vd);

restart = 0; /*reset*/
v4l2core_stop_stream(my_vd);

v4l2core_clean_buffers(my_vd);

/*try new format (values prepared by the request callback)*/
ret = v4l2core_update_current_format(my_vd);
/*try to set the video stream format on the device*/
if(ret != E_OK)
{
fprintf(stderr, “GUCVIEW: could not set the defined stream format\n”);
fprintf(stderr, “GUCVIEW: trying first listed stream format\n”);

v4l2core_prepare_valid_format(my_vd);
v4l2core_prepare_valid_resolution(my_vd);
ret = v4l2core_update_current_format(my_vd);

if(ret != E_OK)
{
fprintf(stderr, “GUCVIEW: also could not set the first listed stream format\n”);

gui_error(“Guvcview error”, “could not start a video stream in the device”, 1);

return ((void *) -1);
}
}

if((current_width != v4l2core_get_frame_width(my_vd)) ||
current_height != v4l2core_get_frame_height(my_vd))
{
if(debug_level > 1)
printf(“GUVCVIEW: resolution changed, reseting render\n”);

/*close render*/
render_close();

/*restart the render with new format*/
if(render_init(
render,
v4l2core_get_frame_width(my_vd),
v4l2core_get_frame_height(my_vd),
render_flags,
my_options->render_width,
my_options->render_height) < 0) render = RENDER_NONE; else { render_set_event_callback(EV_QUIT, &quit_callback, NULL); render_set_event_callback(EV_KEY_V, &key_V_callback, NULL); render_set_event_callback(EV_KEY_I, &key_I_callback, NULL); render_set_event_callback(EV_KEY_UP, &key_UP_callback, NULL); render_set_event_callback(EV_KEY_DOWN, &key_DOWN_callback, NULL); render_set_event_callback(EV_KEY_LEFT, &key_LEFT_callback, NULL); render_set_event_callback(EV_KEY_RIGHT, &key_RIGHT_callback, NULL); } } if(debug_level > 0)
printf(“GUVCVIEW: reset to pixelformat=%x width=%i and height=%i\n”,
v4l2core_get_requested_frame_format(my_vd),
v4l2core_get_frame_width(my_vd),
v4l2core_get_frame_height(my_vd));

v4l2core_start_stream(my_vd);

}

/*get the frame from v4l2 core*/
frame = v4l2core_get_decoded_frame(my_vd);
if( frame != NULL)
{
int i =0;
uint8_t *tmp=frame->yuv_frame;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}
/*run software autofocus (must be called after frame was grabbed and decoded)*/
if(do_soft_autofocus || do_soft_focus)
do_soft_focus = v4l2core_soft_autofocus_run(my_vd, frame);

/* apply fx effects to the frame
* do it before saving the frame
* (we want to store the effects)
*/
render_frame_fx(frame->yuv_frame, my_render_mask);

/*check the timers*/
if(check_photo_timer())
{
if((frame->timestamp – my_last_photo_time) > my_photo_timer)
{
save_image = 1;
my_last_photo_time = frame->timestamp;

if(my_options->photo_npics > 0)
{
if(my_photo_npics > 0)
my_photo_npics–;
else
{
save_image = 0;
stop_photo_timer(); /*close timer*/
if(!check_video_timer() && my_options->exit_on_term > 0)
quit_callback(NULL); /*close app*/
}
}
}
}

if(check_video_timer())
{
if((frame->timestamp – my_video_begin_time) > my_video_timer)
{
stop_video_timer();
if(!check_photo_timer() && my_options->exit_on_term > 0)
quit_callback(NULL); /*close app*/
}
}

/*save the frame (photo)*/
if(save_image)
{
char *img_filename = NULL;

/*get_photo_[name|path] always return a non NULL value*/
char *name = strdup(get_photo_name());
char *path = strdup(get_photo_path());

if(get_photo_sufix_flag())
{
char *new_name = add_file_suffix(path, name);
free(name); /*free old name*/
name = new_name; /*replace with suffixed name*/
}
int pathsize = strlen(path);
if(path[pathsize – 1] != ‘/’)
img_filename = smart_cat(path, ‘/’, name);
else
img_filename = smart_cat(path, 0, name);

//if(debug_level > 1)
// printf(“GUVCVIEW: saving image to %s\n”, img_filename);

snprintf(status_message, 79, _(“saving image to %s”), img_filename);
gui_status_message(status_message);

v4l2core_save_image(frame, img_filename, get_photo_format());

free(path);
free(name);
free(img_filename);

save_image = 0; /*reset*/
}

/*save the frame (video)*/
if(video_capture_get_save_video())
{
int size = (frame->width * frame->height * 3) / 2;

uint8_t *input_frame = frame->yuv_frame;
/*
* TODO: check codec_id, format and frame flags
* (we may want to store a compressed format
*/
if(get_video_codec_ind() == 0) //raw frame
{
switch(v4l2core_get_requested_frame_format(my_vd))
{
case V4L2_PIX_FMT_H264:
input_frame = frame->h264_frame;
size = (int) frame->h264_frame_size;
break;
default:
input_frame = frame->raw_frame;
size = (int) frame->raw_frame_size;
break;
}

}
/*add the frame to the encoder buffer*/
encoder_add_video_frame(input_frame, size, frame->timestamp, frame->isKeyframe);

/*
* exponencial scheduler
* with 50% threshold (milisec)
* and max value of 250 ms (4 fps)
*/
double time_sched = encoder_buff_scheduler(ENCODER_SCHED_LIN, 0.5, 250);
if(time_sched > 0)
{
switch(v4l2core_get_requested_frame_format(my_vd))
{
case V4L2_PIX_FMT_H264:
{
uint32_t framerate = lround(time_sched * 1E6); /*nanosec*/
v4l2core_set_h264_frame_rate_config(my_vd, framerate);
break;
}
default:
{
struct timespec req = {
.tv_sec = 0,
.tv_nsec = (uint32_t) time_sched * 1E6};/*nanosec*/
nanosleep(&req, NULL);
break;
}
}
}
}

/* render the osd
* must be done after saving the frame
* (we don’t want to record the osd effects)
*/
render_frame_osd(frame->yuv_frame);

/* finally render the frame */
snprintf(render_caption, 29, “Guvcview (%2.2f fps)”,
v4l2core_get_realfps(my_vd));
render_set_caption(render_caption);
render_frame(frame->yuv_frame);

/*we are done with the frame buffer release it*/
v4l2core_release_frame(my_vd, frame);
}
}

v4l2core_stop_stream(my_vd);

/*if we are still saving video then stop it*/
if(video_capture_get_save_video())
stop_encoder_thread();

render_close();

return ((void *) 0);
}

You can see in the last set part of the function they call yuv_frame, so I modified frame->yuv_frame and noticed that it impacted the image by chopping off part of the image!

At this point I got frustrated because it was getting hard to track everything so I looked at the log again and I saw the following information in the verbose output:

V4L2_CORE: process frame queue index 0
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7f2ef9203010

That lead me to frame_decoder.c in which I found the following code (edit is in bold)

/*
* decode video stream ( from raw_frame to frame buffer (yuyv format))
* args:
* vd – pointer to device data
* frame – pointer to frame buffer
*
* asserts:
* vd is not null
*
* returns: error code ( 0 – E_OK)
*/
int decode_v4l2_frame(v4l2_dev_t *vd, v4l2_frame_buff_t *frame)
{
/*asserts*/
assert(vd != NULL);

if(!frame->raw_frame || frame->raw_frame_size == 0)
{
fprintf(stderr, “V4L2_CORE: not decoding empty raw frame (frame of size %i at 0x%p)\n”, (int) frame->raw_frame_size, frame->raw_frame);
return E_DECODE_ERR;
}

if(verbosity > 3)
printf(“V4L2_CORE: decoding raw frame of size %i at 0x%p\n”,
(int) frame->raw_frame_size, frame->raw_frame );

int ret = E_OK;

int width = vd->format.fmt.pix.width;
int height = vd->format.fmt.pix.height;

frame->isKeyframe = 0; /*reset*/

/*
* use the requested format since it may differ
* from format.fmt.pix.pixelformat (muxed H264)
*/
int format = vd->requested_fmt;

int framesizeIn =(width * height << 1);//2 bytes per pixel
printf(“the format is %d\n”, format);
// able to change raw video upstream of switch statement!!”);
int i =0;
uint8_t *tmp=frame->raw_frame;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}

switch (format)
{
case V4L2_PIX_FMT_H264:
/*
* get the h264 frame in the tmp_buffer
*/
frame->h264_frame_size = demux_h264(
frame->h264_frame,
frame->raw_frame,
frame->raw_frame_size,
frame->h264_frame_max_size);

/*
* store SPS and PPS info (usually the first two NALU)
* and check/store the last IDR frame
*/
store_extra_data(vd, frame);

/*
* check for keyframe and store it
*/
frame->isKeyframe = is_h264_keyframe(vd, frame);

//decode if we already have a IDR frame
if(vd->h264_last_IDR_size > 0)
{
/*no need to convert output*/
h264_decode(frame->yuv_frame, frame->h264_frame, frame->h264_frame_size);
}
break;

case V4L2_PIX_FMT_JPEG:
case V4L2_PIX_FMT_MJPEG:
if(frame->raw_frame_size <= HEADERFRAME1) { // Prevent crash on empty image fprintf(stderr, “V4L2_CORE: (jpeg decoder) Ignoring empty buffer\n”); ret = E_DECODE_ERR; return (ret); } ret = jpeg_decode(frame->yuv_frame, frame->raw_frame, frame->raw_frame_size);

This resulted in the following image:

Screenshot from 2019-05-10 19-05-52

This step is pretty important because now I am able to modify the raw frame image and which will then get converted to yuv but I can start from a lower level. I know the image was impacted because of the green hue!

I then looked at where this function is called with the following command:

xx:~/git-repos/guvcview$ grep -r –include=”*.*” “decode_v4l2_frame”
gview_v4l2core/frame_decoder.c:int decode_v4l2_frame(v4l2_dev_t *vd, v4l2_frame_buff_t *frame)
gview_v4l2core/frame_decoder.h:int decode_v4l2_frame(v4l2_dev_t *vd, v4l2_frame_buff_t *frame);
Binary file gview_v4l2core/libgviewv4l2core_la-frame_decoder.o matches
Binary file gview_v4l2core/.libs/libgviewv4l2core.a matches
Binary file gview_v4l2core/.libs/libgviewv4l2core_la-frame_decoder.o matches
Binary file gview_v4l2core/.libs/libgviewv4l2core-2.0.so.2.0.0 matches
Binary file gview_v4l2core/.libs/libgviewv4l2core_la-v4l2_core.o matches
Binary file gview_v4l2core/libgviewv4l2core_la-v4l2_core.o matches

gview_v4l2core/v4l2_core.c: if(decode_v4l2_frame(vd, frame) != E_OK)

As you can see there is a new file that shows up! The new file is v4l2_core.c, I hope this leads to better places!!

I then searched for the function decode_v4l2_frame and found the following piece of code:

/*
* gets the next video frame and decodes it
* args:
* vd – pointer to v4l2 device handler
*
* returns: pointer to decoded frame buffer ( NULL on error)
*/
v4l2_frame_buff_t *v4l2core_get_decoded_frame(v4l2_dev_t *vd)
{
v4l2_frame_buff_t *frame = v4l2core_get_frame(vd);
if(frame != NULL)
{
/*decode the raw frame*/
if(decode_v4l2_frame(vd, frame) != E_OK)
{
fprintf(stderr, “V4L2_CORE: Error – Couldn’t decode frame\n”);
}
}

return frame;
}

What is interesting v4l2core_get_decoded_frame is called and here is what I found (edits are in bold):

/*
* gets the next video frame (must be released after processing)
* args:
* vd – pointer to v4l2 device handler
*
* asserts:
* vd is not null
*
* returns: pointer frame buffer (NULL on error)
*/
v4l2_frame_buff_t *v4l2core_get_frame(v4l2_dev_t *vd)
{
/*asserts*/
assert(vd != NULL);

/*for H264 streams request a IDR frame with SPS and PPS data if it’s the first frame*/
if(vd->requested_fmt == V4L2_PIX_FMT_H264 && vd->frame_index < 1)
request_h264_frame_type(vd, PICTURE_TYPE_IDR_FULL);

int res = 0;
int ret = check_frame_available(vd);

int qind = -1;

if (ret < 0)
return NULL;

int bytes_used = 0;
printf(“vd->mutex %p\n”,__PMUTEX);

switch(vd->cap_meth)
{
case IO_READ:

/*lock the mutex*/
printf(“IO_READ\n”);
__LOCK_MUTEX( __PMUTEX );
if(vd->streaming == STRM_OK)
{
vd->buf.bytesused = v4l2_read (vd->fd, vd->mem[vd->buf.index], vd->buf.length);
bytes_used = vd->buf.bytesused;

if(bytes_used > 0)
qind = process_input_buffer(vd);
}
else res = -1;
/*unlock the mutex*/
__UNLOCK_MUTEX( __PMUTEX );
if (-1==bytes_used)
return NULL;

if(res < 0)
return NULL;

if (-1 == bytes_used )
{
printf(“inside the bytes_used if statement\n”);
switch (errno)
{
case EAGAIN:
fprintf(stderr, “V4L2_CORE: No data available for read: %s\n”, strerror(errno));
break;
case EINVAL:
fprintf(stderr, “V4L2_CORE: Read method error, try mmap instead: %s\n”, strerror(errno));
break;
case EIO:
fprintf(stderr, “V4L2_CORE: read I/O Error: %s\n”, strerror(errno));
break;
default:
fprintf(stderr, “V4L2_CORE: read: %s\n”, strerror(errno));
break;
}
return NULL;
}

break;

case IO_MMAP:
default:
//if((vd->setH264ConfigProbe > 0))
//{

//if(vd->setH264ConfigProbe)
//{
//video_disable(vd);
//unmap_buff();

//h264_commit(vd, global);

//vd->setH264ConfigProbe = 0;
//query_buff(vd);
//queue_buff(vd);
//video_enable(vd);
//}

//ret = check_frame_available(vd);

//if (ret < 0)
//return ret;
//}

/* dequeue the buffers */

printf(“IO_MMAP\n”);
/*lock the mutex*/
__LOCK_MUTEX( __PMUTEX );

if(vd->streaming == STRM_OK)
{
printf(“STRM_OK\n”);
memset(&vd->buf, 0, sizeof(struct v4l2_buffer));

vd->buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
vd->buf.memory = V4L2_MEMORY_MMAP;

ret = xioctl(vd->fd, VIDIOC_DQBUF, &vd->buf);

if(!ret)
qind = process_input_buffer(vd);
else
fprintf(stderr, “V4L2_CORE: (VIDIOC_DQBUF) Unable to dequeue buffer: %s\n”, strerror(errno));
}
else res = -1;

/*unlock the mutex*/
__UNLOCK_MUTEX( __PMUTEX );

if(res < 0 || ret < 0)
return NULL;
}

if(qind < 0 || qind >= vd->frame_queue_size)
return NULL;

vd->frame_queue[qind].width = vd->format.fmt.pix.width;
vd->frame_queue[qind].height = vd->format.fmt.pix.height;

return &vd->frame_queue[qind];
}

As you can see I created a lot of print statements to see which piece of code was being used and see if I could manipulate it.

Here was output in the log:

IO_READ
GUVCVIEW: joining capture thread
In 2nd if statement
In else statement
bytes_used (result) 4294967295
buf->sequence 0
buf->memory 1
vd->mutex 0x55d933e675a0
IO_READ
In else statement
bytes_used (result) 614400
buf->sequence 0
buf->memory 1
The if statement inside the else
V4L2_CORE: process frame queue index 0
V4L2CORE: (fps) ref:0 ts:445941891611482 frames:1
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7f7795ece010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x55d933e675a0

As you can see I am using IO_READ and using the 2nd if statement which is important.

I was able to modify the image by the following code (represented in bold);

switch(vd->cap_meth)
{
case IO_READ:

/*lock the mutex*/
printf(“IO_READ\n”);
__LOCK_MUTEX( __PMUTEX );
if(vd->streaming == STRM_OK)
{
vd->buf.bytesused = v4l2_read (vd->fd, vd->mem[vd->buf.index], vd->buf.length);
bytes_used = vd->buf.bytesused;

if(bytes_used > 0)
qind = process_input_buffer(vd);
}
else res = -1;
/*unlock the mutex*/
__UNLOCK_MUTEX( __PMUTEX );
if (-1==bytes_used)
return NULL;
int i =0;
uint8_t *tmp=vd->frame_queue[qind].raw_frame;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}

This was added to v4l2core_get_frame function.

With the change I was once again able modify the raw image.

Since I was inside IO_READ, I decided to see where vd is coming from and I saw the function v4l2_read that is being called. I did the following command to see where v4l2_read is coming from:

0xxx:~$ grep -r –include=”*.c” “v4l2_read”
guvcview-src-2.0.6_mod/gview_v4l2core/v4l2_core.c: vd->buf.bytesused = v4l2_read (vd->fd, vd->mem[vd->buf.index], vd->buf.length);
grep: .dbus: Permission denied
git-repos/guvcview/gview_v4l2core/v4l2_core.c: vd->buf.bytesused = v4l2_read (vd->fd, vd->mem[vd->buf.index], vd->buf.length);
git/kernels/staging/drivers/media/usb/hdpvr/hdpvr-video.c: * hdpvr_v4l2_read()
git/kernels/staging/drivers/media/usb/pvrusb2/pvrusb2-v4l2.c:static ssize_t pvr2_v4l2_read(struct file *file,
git/kernels/staging/drivers/media/usb/pvrusb2/pvrusb2-v4l2.c: .read = pvr2_v4l2_read,
git/kernels/staging/drivers/media/usb/usbvision/usbvision-video.c:static ssize_t usbvision_v4l2_read(struct file *file, char __user *buf,
git/kernels/staging/drivers/media/usb/usbvision/usbvision-video.c: .read = usbvision_v4l2_read,
git/kernels/staging/drivers/media/usb/cx231xx/cx231xx-video.c: * cx231xx_v4l2_read()
git/kernels/staging/drivers/media/usb/cx231xx/cx231xx-video.c:cx231xx_v4l2_read(struct file *filp, char __user *buf, size_t count,
git/kernels/staging/drivers/media/usb/cx231xx/cx231xx-video.c: .read = cx231xx_v4l2_read,
git/kernels/staging/drivers/media/usb/uvc/uvc_v4l2.c:static ssize_t uvc_v4l2_read(struct file *file, char __user *data,
git/kernels/staging/drivers/media/usb/uvc/uvc_v4l2.c: uvc_trace(UVC_TRACE_CALLS, “uvc_v4l2_read: not implemented.\n”);
git/kernels/staging/drivers/media/usb/uvc/uvc_v4l2.c: .read = uvc_v4l2_read,
git/kernels/staging/drivers/media/v4l2-core/v4l2-dev.c:static ssize_t v4l2_read(struct file *filp, char __user *buf,
git/kernels/staging/drivers/media/v4l2-core/v4l2-dev.c: .read = v4l2_read,
git/kernels/staging/drivers/media/pci/cx18/cx18-fileops.c:ssize_t cx18_v4l2_read(struct file *filp, char __user *buf, size_t count,
git/kernels/staging/drivers/media/pci/cx18/cx18-streams.c: .read = cx18_v4l2_read,
git/kernels/staging/drivers/media/pci/ivtv/ivtv-fileops.c:ssize_t ivtv_v4l2_read(struct file * filp, char __user *buf, size_t count, loff_t * pos)
git/kernels/staging/drivers/media/pci/ivtv/ivtv-streams.c: .read = ivtv_v4l2_read,
git/kernels/staging/drivers/media/pci/ivtv/ivtv-streams.c: .read = ivtv_v4l2_read,
v4l-utils/contrib/test/sdlcam.c: int num = v4l2_read(dev.fd, dev.buf, SIZE);
v4l-utils/lib/libv4l2/libv4l2.c:static int v4l2_read_and_convert(int index, unsigned char *dest, int dest_size)
v4l-utils/lib/libv4l2/libv4l2.c:ssize_t v4l2_read(int fd, void *dest, size_t n)
v4l-utils/lib/libv4l2/libv4l2.c: /* The read call done by v4l2_read_and_convert will start the stream */
v4l-utils/lib/libv4l2/libv4l2.c: result = v4l2_read_and_convert(index, dest, n);
v4l-utils/lib/libv4l2/v4l2convert.c: return v4l2_read(fd, buffer, n);
v4l-utils/lib/libv4l1/libv4l1.c: result = v4l2_read(devices[index].fd,
v4l-utils/lib/libv4l1/libv4l1.c: result = v4l2_read(fd, buffer, n);

This lead me to the file that is underlined because I didn’t want to go into the kernel just yet!

So right now I am at libv4l2.c.

I will have more for the next blog!

Advertisements

Trying to understand UVC Part 4

Ok, Since the last time I was at the function:

int v4lconvert_convert

which is inside libv4lconvert.c. When I looked more into the code I couldn’t find anything use to go from. It seemed that this code was only used for converting something. I wasn’t able to track the raw data to its source.

The most interesting part was the memcopy function, but I couldn’t go any futher. I wanted to know where src was coming from!

I decided to go back to libv4l2.c and starting going back through the v4l2_read function. My additions are in bold:

ssize_t v4l2_read(int fd, void *dest, size_t n)
{
ssize_t result;
int saved_errno;
int index = v4l2_get_index(fd);

if (index == -1)
return SYS_READ(fd, dest, n);

if (!devices[index].dev_ops->read) {
errno = EINVAL;
return -1;
}
printf(“INSiDE V4L2_READ! -horrible\n”);

pthread_mutex_lock(&devices[index].stream_lock);

/* When not converting and the device supports read(), let the kernel handle
it */
if (devices[index].convert == NULL ||
((devices[index].flags & V4L2_SUPPORTS_READ) &&
!v4l2_needs_conversion(index))) {
result = devices[index].dev_ops->read(
devices[index].dev_ops_priv,
fd, dest, n);
printf(“In 1st if statement\n”);
goto leave;
}

/* Since we need to do conversion try to use mmap (streaming) mode under
the hood as that safes a memcpy for each frame read.

Note sometimes this will fail as some drivers (at least gspca) do not allow
switching from read mode to mmap mode and they assume read() mode if a
select or poll() is done before any buffers are requested. So using mmap
mode under the hood will fail if a select() or poll() is done before the
first emulated read() call. */
if (!(devices[index].flags & V4L2_STREAM_CONTROLLED_BY_READ) &&
!(devices[index].flags & V4L2_USE_READ_FOR_READ)) {
result = v4l2_activate_read_stream(index);
printf(“In 2nd if statement,activating read stream\n”);
if (result) {
printf(“activating_read_stream failed us read() instead\n”);
/* Activating mmap mode failed, use read() instead */
devices[index].flags |= V4L2_USE_READ_FOR_READ;
/* The read call done by v4l2_read_and_convert will start the stream */
devices[index].first_frame = V4L2_IGNORE_FIRST_FRAME_ERRORS;
}
}

if (devices[index].flags & V4L2_USE_READ_FOR_READ) {
result = v4l2_read_and_convert(index, dest, n);
printf(“In 3rd if statement \n”);
} else {
printf(“In else statement with 3rd if\n”);
struct v4l2_buffer buf;

buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
result = v4l2_dequeue_and_convert(index, &buf, dest, n);
printf(“buf addres %p\n”,&buf);
printf(“bytes_used (result) %lu\n”,result);
printf(“buf->sequence 0x%x\n”,buf.sequence);
printf(“buf->memory 0x%x\n”,buf.memory);
printf(“buf->offset 0x%x\n”,buf.m.offset);
printf(“index %d\n”, index);
printf(“n: %lx\n”,n);
printf(“buf->v4l2_plane %p\n”,buf.m.planes);
//I can’t get anything to happen with the below if statment. It seems as if
//buf.m.planes has nothing populated and I keep getting errors
/*if((buf.m.planes!=NULL) && (buf.m.planes->bytesused>0)) {
printf(“inside my own if statement\n”);
printf(“buf.m.planes->bytesused %i\n”,buf.m.planes->bytesused);
//printf(“buf.m.planes->m.mem_offset 0x%x\n”,buf.m.planes->m.mem_offset);
//printf(“buf.m.planes->m.userptr 0x%lx\n”,buf.m.planes->m.userptr);
//printf(“buf.m.planes->data_offset 0x%x\n”,buf.m.planes->data_offset);
}*/
printf(“buf->userptr %lx\n”,buf.m.userptr);
printf(“buf.m.length %i\n”,buf.length);
printf(“buf.type %x\n”,buf.type);
printf(“buf.flags %x\n”,buf.flags);
//this seems to be giving a segmentation fault
/*if(buf.m.userptr) {
if((result<1000000) && (result>0)){
int i =0;
uint8_t *tmp=(uint8_t *)(buf.memory);
for(i=0;i<2;i++) {
*tmp=0;
tmp++;
}
}
}*/

//printf(“dest->memory %u\n”,dest.memory);

if (result >= 0) {
printf(“The if statement inside the else\n”);
v4l2_queue_read_buffer(index, buf.index);

printf(“devices[index].frame_pointers[0] %02x\n”,(unsigned int)devices[index].frame_pointers[0]);
}
}

leave:
saved_errno = errno;
pthread_mutex_unlock(&devices[index].stream_lock);
errno = saved_errno;
/*if(result > 0) {
int i =0;
uint8_t *tmp=(uint8_t *)dest;
printf(“dest address: %p\n”,dest);
printf(“v4l2_read frame: “);
for(i=0;i<3000;i++) {
//printf(“%d,”,*tmp);
*tmp=0;
tmp++;
}
printf(“\n”);
}*/

return result;
}

I started looking into the if else statement first and into the following function:

static int v4l2_queue_read_buffer(int index, int buffer_index)
{
int result;
struct v4l2_buffer buf;

if (devices[index].frame_queued & (1 << buffer_index))
return 0;

memset(&buf, 0, sizeof(buf));
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
buf.index = buffer_index;
result = devices[index].dev_ops->ioctl(devices[index].dev_ops_priv,
devices[index].fd, VIDIOC_QBUF, &buf);
if (result) {
int saved_err = errno;

V4L2_PERROR(“queuing buf %d”, buffer_index);
errno = saved_err;
return result;
}

devices[index].frame_queued |= 1 << buffer_index;

return 0;
}

Now the only useful thing I saw was the following lines:

result = devices[index].dev_ops->ioctl(devices[index].dev_ops_priv,
devices[index].fd, VIDIOC_QBUF, &buf);

I did the following search :

x:~$ grep -r –include “*.c” ioctl | wc -l
grep: .dbus: Permission denied
15383

15383 is way too many results to to look through.

So i decided to look into what dev_ops meant. I stareted with looking into what the devices variable was:

static struct v4l2_dev_info devices[V4L2_MAX_DEVICES] = {
{ .fd = -1 },
{ .fd = -1 }, { .fd = -1 }, { .fd = -1 }, { .fd = -1 }, { .fd = -1 },
{ .fd = -1 }, { .fd = -1 }, { .fd = -1 }, { .fd = -1 }, { .fd = -1 },
{ .fd = -1 }, { .fd = -1 }, { .fd = -1 }, { .fd = -1 }, { .fd = -1 }
};

So the struct here is v4l2_dev_info was found with the following command:

xxx:~$ grep -r –include “*.h” “v4l2_dev_info”
grep: .dbus: Permission denied
v4l-utils/lib/libv4l2/libv4l2-priv.h: the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
v4l-utils/lib/libv4l2/libv4l2-priv.h:struct v4l2_dev_info {

This lead to the following program:

struct v4l2_dev_info {
int fd;
int flags;
int open_count;
int gone; /* Set to 1 when a device is detached (ENODEV encountered) */
long page_size;
/* actual format of the cam */
struct v4l2_format src_fmt;
/* fmt as seen by the application (iow after conversion) */
struct v4l2_format dest_fmt;
pthread_mutex_t stream_lock;
unsigned int no_frames;
unsigned int nreadbuffers;
int fps;
int first_frame;
struct v4lconvert_data *convert;
unsigned char *convert_mmap_buf;
size_t convert_mmap_buf_size;
size_t convert_mmap_frame_size;
/* Frame bookkeeping is only done when in read or mmap-conversion mode */
unsigned char *frame_pointers[V4L2_MAX_NO_FRAMES];
int frame_sizes[V4L2_MAX_NO_FRAMES];
int frame_queued; /* 1 status bit per frame */
int frame_info_generation;
/* mapping tracking of our fake (converting mmap) frame buffers */
unsigned char frame_map_count[V4L2_MAX_NO_FRAMES];
/* buffer when doing conversion and using read() for read() */
int readbuf_size;
unsigned char *readbuf;
/* plugin info */
void *plugin_library;
void *dev_ops_priv;
const struct libv4l_dev_ops *dev_ops;
};

The last line has dev_ops which is another struct. Now time to find that struct and here was my results:

xxx:~$ grep -r –include “*.h” “libv4l_dev_ops”
grep: .dbus: Permission denied
v4l-utils/lib/libv4lconvert/control/libv4lcontrol.h: const struct libv4l_dev_ops *dev_ops, int always_needs_conversion);
v4l-utils/lib/libv4lconvert/control/libv4lcontrol-priv.h: const struct libv4l_dev_ops *dev_ops;
v4l-utils/lib/libv4lconvert/libv4lconvert-priv.h: const struct libv4l_dev_ops *dev_ops;
v4l-utils/lib/include/libv4lconvert.h:struct libv4l_dev_ops;
v4l-utils/lib/include/libv4lconvert.h:LIBV4L_PUBLIC const struct libv4l_dev_ops *v4lconvert_get_default_dev_ops();
v4l-utils/lib/include/libv4lconvert.h: void *dev_ops_priv, const struct libv4l_dev_ops *dev_ops);
v4l-utils/lib/include/libv4l-plugin.h:/* Structure libv4l_dev_ops holds the calls from libv4ls to video nodes.
v4l-utils/lib/include/libv4l-plugin.h:struct libv4l_dev_ops {
v4l-utils/lib/libv4l2/libv4l2-priv.h: const struct libv4l_dev_ops *dev_ops;
v4l-utils/lib/libv4l2/libv4l2-priv.h: const struct libv4l_dev_ops **dev_ops_ret);
v4l-utils/lib/libv4l2/libv4l2-priv.h: const struct libv4l_dev_ops *dev_ops);
v4l-utils/lib/libv4l2/libv4l2-priv.h: const struct libv4l_dev_ops **dev_ops_ret)
v4l-utils/lib/libv4l2/libv4l2-priv.h: const struct libv4l_dev_ops *dev_ops)

I looked int libv4l2-priv.h and I didn’t find anything useful but when I went to libv4l-plugin.h I found the following program:

#ifndef __LIBV4L_PLUGIN_H
#define __LIBV4L_PLUGIN_H

#include <sys/types.h>

/* Structure libv4l_dev_ops holds the calls from libv4ls to video nodes.
They can be normal open/close/ioctl etc. or any of them may be replaced
with a callback by a loaded plugin.
*/

struct libv4l_dev_ops {
void * (*init)(int fd);
void (*close)(void *dev_ops_priv);
int (*ioctl)(void *dev_ops_priv, int fd, unsigned long int request, void *arg);
ssize_t (*read)(void *dev_ops_priv, int fd, void *buffer, size_t n);
ssize_t (*write)(void *dev_ops_priv, int fd, const void *buffer, size_t n);
/* For future plugin API extension, plugins implementing the current API
must set these all to NULL, as future versions may check for these */
void (*reserved1)(void);
void (*reserved2)(void);
void (*reserved3)(void);
void (*reserved4)(void);
void (*reserved5)(void);
void (*reserved6)(void);
void (*reserved7)(void);
};

#endif

Great so I found where ioctl is defined, but I have no I idea how the function is defined…

So I go back to libv4l2.c and look for ioctl function there

I find the following function thinking it’s somehow connected and see:

int v4l2_ioctl(int fd, unsigned long int request, …)
{
printf(“Inside v4l2_ioctl \n”);
void *arg;
va_list ap;
int result, index, saved_err;
int is_capture_request = 0, stream_needs_locking = 0;

va_start(ap, request);
arg = va_arg(ap, void *);
va_end(ap);

index = v4l2_get_index(fd);
if (index == -1)
return SYS_IOCTL(fd, request, arg);

/* Apparently the kernel and / or glibc ignore the 32 most significant bits
when long = 64 bits, and some applications pass an int holding the req to
ioctl, causing it to get sign extended, depending upon this behavior */
request = (unsigned int)request;

I look for my print statement in bold and I see the following line:

AUDIO: Portaudio device changed to 4
V4L2_CORE: checking format: YUYV
Inside v4l2_ioctl
inside v4l2_buffer_mapped

V4L2_CORE: allocating frame buffers
V4L2_CORE: trying to change fps to 1/25
Inside v4l2_ioctl
Inside v4l2_ioctl
inside v4l2_buffer_mapped

You also see I printed out v4l2_buffer_mapped function which is also in libv4l2.c I still don’t see the Inside v4l2_ioctl repeating everytime a new frame is captured which is. So where is this function defined?

So the (*ioctl) is a function pointer and I have to find where it’s defined.

I found the following file after looking for where dev_ops was and here is it where it lead me to:

static const struct libv4l_dev_ops default_dev_ops = {
.init = dev_init,
.close = dev_close,
.ioctl = dev_ioctl,
.read = dev_read,
.write = dev_write,
};

Ok so there is a function called dev_ioctl and I found it in the same function:

static int dev_ioctl(void *dev_ops_priv, int fd, unsigned long cmd, void *arg)
{
printf(“inside dev_ioctl of libv4lconvert -horrible\n\n\n\n”);
return SYS_IOCTL(fd, cmd, arg);
}

If this function is true then when I print I should see this code constantly show up (bold is my changes in above code block).

Once I made the change I saw the following output:

devices[index].frame_pointers[0] bece000
V4L2_CORE: process frame queue index 0
raw data location: 0x7fe126bd3010
process_queue_frame (vd->mem) frame: 40,134,42,126,39,127,37,129,34,134,34,128,33,132,34,127,34,137,35,128,38,137,40,129,42,130,42,128,39,136,
frame data: 40,134,42,126,39,127,37,129,34,134,34,128,33,132,34,127,34,137,35,128,38,137,40,129,42,130,42,128,39,136,
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7fe126bd3010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x55e96a3185a0
IO_READ
INSiDE V4L2_READ! -horrible
In else statement with 3rd if
inside v4l2_dequeue_and_convert
inside dev_ioctl of libv4lconvert -horrible

v4lconvert_convert
No conversion needed!!
buf addres 0x7fe10ba6c8e0
bytes_used (result) 614400
buf->sequence 0xe
buf->memory 0x1
buf->offset 0x12c000
index 0
n: e1000
buf->v4l2_plane 0x55e90012c000
buf->userptr 55e90012c000
buf.m.length 614400
buf.type 1
buf.flags 12001
The if statement inside the else
inside dev_ioctl of libv4lconvert -horrible

devices[index].frame_pointers[0] bece000
V4L2_CORE: process frame queue index 0
raw data location: 0x7fe126bd3010
process_queue_frame (vd->mem) frame: 38,137,36,127,34,131,33,130,33,139,34,128,34,132,36,128,37,128,38,129,40,135,38,126,40,127,41,128,43,138,
frame data: 38,137,36,127,34,131,33,130,33,139,34,128,34,132,36,128,37,128,38,129,40,135,38,126,40,127,41,128,43,138,
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7fe126bd3010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x55e96a3185a0
IO_READ
INSiDE V4L2_READ! -horrible
In else statement with 3rd if
inside v4l2_dequeue_and_convert
inside dev_ioctl of libv4lconvert -horrible

v4lconvert_convert
No conversion needed!!
buf addres 0x7fe10ba6c8e0
bytes_used (result) 614400
buf->sequence 0xf
buf->memory 0x1
buf->offset 0x1c2000
index 0
n: e1000
buf->v4l2_plane 0x55e9001c2000
buf->userptr 55e9001c2000
buf.m.length 614400
buf.type 1
buf.flags 12001
The if statement inside the else
inside dev_ioctl of libv4lconvert -horrible

devices[index].frame_pointers[0] bece000
V4L2_CORE: process frame queue index 0
raw data location: 0x7fe126bd3010
process_queue_frame (vd->mem) frame: 33,130,35,129,37,130,36,126,37,129,38,128,39,128,37,126,38,133,41,128,43,138,44,127,43,131,43,127,43,129,
frame data: 33,130,35,129,37,130,36,126,37,129,38,128,39,128,37,126,38,133,41,128,43,138,44,127,43,131,43,127,43,129,
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7fe126bd3010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x55e96a3185a0
IO_READ
INSiDE V4L2_READ! -horrible
In else statement with 3rd if
inside v4l2_dequeue_and_convert
inside dev_ioctl of libv4lconvert -horrible

v4lconvert_convert
No conversion needed!!
buf addres 0x7fe10ba6c8e0
bytes_used (result) 614400
buf->sequence 0x10
buf->memory 0x1
buf->offset 0x0
index 0
n: e1000
buf->v4l2_plane 0x55e900000000
buf->userptr 55e900000000
buf.m.length 614400
buf.type 1
buf.flags 12001
The if statement inside the else
inside dev_ioctl of libv4lconvert -horrible

devices[index].frame_pointers[0] bece000
V4L2_CORE: process frame queue index 0
raw data location: 0x7fe126bd3010
process_queue_frame (vd->mem) frame: 38,129,37,128,38,127,39,129,40,127,42,127,45,142,46,125,45,134,44,128,44,129,43,127,43,127,43,130,42,138,
frame data: 38,129,37,128,38,127,39,129,40,127,42,127,45,142,46,125,45,134,44,128,44,129,43,127,43,127,43,130,42,138,
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7fe126bd3010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x55e96a3185a0
IO_READ
INSiDE V4L2_READ! -horrible
In else statement with 3rd if
inside v4l2_dequeue_and_convert
inside dev_ioctl of libv4lconvert -horrible

So now I have seen that dev_ioctl is most likely dev_ops->ioctl

Now what is in that function?

I search for the SYS_IOCTL and found libv4lsyscall-priv.h

/*-
* Copyright (c) 2009 Hans Petter Selasky. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS “AS IS” AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/

/*
* The following file allows for having the complete V4L stack and
* hardware drivers in userland.
*/

#ifndef _LIBV4LSYSCALL_PRIV_H_
#define _LIBV4LSYSCALL_PRIV_H_

/* Some of these headers are not needed by us, but by linux/videodev2.h,
which is broken on some systems and doesn’t include them itself 😦 */

#ifdef linux
#include <sys/time.h>
#include <sys/syscall.h>
#include <linux/types.h>
#include <linux/ioctl.h>
/* On 32 bits archs we always use mmap2, on 64 bits archs there is no mmap2 */
#ifdef __NR_mmap2
#if !defined(SYS_mmap2)
#define SYS_mmap2 __NR_mmap2
#endif
#define MMAP2_PAGE_SHIFT 12
#else
#define SYS_mmap2 SYS_mmap
#define MMAP2_PAGE_SHIFT 0
#endif
#endif

#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
#include <sys/time.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#define _IOC_NR(cmd) ((cmd) & 0xFF)
#define _IOC_TYPE(cmd) IOCGROUP(cmd)
#define _IOC_SIZE(cmd) IOCPARM_LEN(cmd)
#define MAP_ANONYMOUS MAP_ANON
#define MMAP2_PAGE_SHIFT 0
#endif

#if defined(__OpenBSD__)
#include <sys/syscall.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#define _IOC_NR(cmd) ((cmd) & 0xFF)
#define _IOC_TYPE(cmd) IOCGROUP(cmd)
#define MMAP2_PAGE_SHIFT 0
#endif

#undef SYS_OPEN
#undef SYS_CLOSE
#undef SYS_IOCTL
#undef SYS_READ
#undef SYS_WRITE
#undef SYS_MMAP
#undef SYS_MUNMAP

#ifndef CONFIG_SYS_WRAPPER

#ifdef SYS_openat
#define SYS_OPEN(file, oflag, mode) \
syscall(SYS_openat, AT_FDCWD, (const char *)(file), (int)(oflag), (mode_t)(mode))
#else
#define SYS_OPEN(file, oflag, mode) \
syscall(SYS_open, (const char *)(file), (int)(oflag), (mode_t)(mode))
#endif
#define SYS_CLOSE(fd) \
syscall(SYS_close, (int)(fd))
#define SYS_IOCTL(fd, cmd, arg) \
syscall(SYS_ioctl, (int)(fd), (unsigned long)(cmd), (void *)(arg))
#define SYS_READ(fd, buf, len) \
syscall(SYS_read, (int)(fd), (void *)(buf), (size_t)(len));
#define SYS_WRITE(fd, buf, len) \
syscall(SYS_write, (int)(fd), (const void *)(buf), (size_t)(len));

#if defined(__FreeBSD__)
#define SYS_MMAP(addr, len, prot, flags, fd, off) \
__syscall(SYS_mmap, (void *)(addr), (size_t)(len), \
(int)(prot), (int)(flags), (int)(fd), (off_t)(off))
#elif defined(__FreeBSD_kernel__)
#define SYS_MMAP(addr, len, prot, flags, fd, off) \
syscall(SYS_mmap, (void *)(addr), (size_t)(len), \
(int)(prot), (int)(flags), (int)(fd), (off_t)(off))
#elif defined(__OpenBSD__)
register_t __syscall(quad_t, …);
#define SYS_MMAP(addr, len, prot, flags, fd, offset) \
__syscall((quad_t)SYS_mmap, (void *)(addr), (size_t)(len), \
(int)(prot), (int)(flags), (int)(fd), 0, (off_t)(offset))
#else
#define SYS_MMAP(addr, len, prot, flags, fd, off) \
syscall(SYS_mmap2, (void *)(addr), (size_t)(len), \
(int)(prot), (int)(flags), (int)(fd), (off_t)((off) >> MMAP2_PAGE_SHIFT))
#endif

#define SYS_MUNMAP(addr, len) \
syscall(SYS_munmap, (void *)(addr), (size_t)(len))

#else

int v4lx_open_wrapper(const char *, int, int);
int v4lx_close_wrapper(int);
int v4lx_ioctl_wrapper(int, unsigned long, void *);
int v4lx_read_wrapper(int, void *, size_t);
int v4lx_write_wrapper(int, const void *, size_t);
void *v4lx_mmap_wrapper(void *, size_t, int, int, int, off_t);
int v4lx_munmap_wrapper(void *, size_t);

#define SYS_OPEN(…) v4lx_open_wrapper(__VA_ARGS__)
#define SYS_CLOSE(…) v4lx_close_wrapper(__VA_ARGS__)
#define SYS_IOCTL(…) v4lx_ioctl_wrapper(__VA_ARGS__)
#define SYS_READ(…) v4lx_read_wrapper(__VA_ARGS__)
#define SYS_WRITE(…) v4lx_write_wrapper(__VA_ARGS__)
#define SYS_MMAP(…) v4lx_mmap_wrapper(__VA_ARGS__)
#define SYS_MUNMAP(…) v4lx_munmap_wrapper(__VA_ARGS__)

#endif

#endif /* _LIBV4LSYSCALL_PRIV_H_ */

So where is v4lx_ioctl_wrapper defined?

 

Trying to understand UVC Part 3

Ok, so I saw that I was able to change libv4lconvert.c.

I looked more into the v4lconvert_convert function and my changes are in bold:

int v4lconvert_convert(struct v4lconvert_data *data,
const struct v4l2_format *src_fmt, /* in */
const struct v4l2_format *dest_fmt, /* in */
unsigned char *src, int src_size, unsigned char *dest, int dest_size)
{
printf(“v4lconvert_convert\n”);
int res, dest_needed, temp_needed, processing, convert = 0;
int rotate90, vflip, hflip, crop;
unsigned char *convert1_dest = dest;
int convert1_dest_size = dest_size;
unsigned char *convert2_src = src, *convert2_dest = dest;
int convert2_dest_size = dest_size;
unsigned char *rotate90_src = src, *rotate90_dest = dest;
unsigned char *flip_src = src, *flip_dest = dest;
unsigned char *crop_src = src;
struct v4l2_format my_src_fmt = *src_fmt;
struct v4l2_format my_dest_fmt = *dest_fmt;
//this impacts the src
/*int i =0;
uint8_t *tmp=src;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}*/

processing = v4lprocessing_pre_processing(data->processing);
rotate90 = data->control_flags & V4LCONTROL_ROTATED_90_JPEG;
hflip = v4lcontrol_get_ctrl(data->control, V4LCONTROL_HFLIP);
vflip = v4lcontrol_get_ctrl(data->control, V4LCONTROL_VFLIP);
crop = my_dest_fmt.fmt.pix.width != my_src_fmt.fmt.pix.width ||
my_dest_fmt.fmt.pix.height != my_src_fmt.fmt.pix.height;

if (/* If no conversion/processing is needed */
(src_fmt->fmt.pix.pixelformat == dest_fmt->fmt.pix.pixelformat &&
!processing && !rotate90 && !hflip && !vflip && !crop) ||
/* or if we should do processing/rotating/flipping but the app tries to
use the native cam format, we just return an unprocessed frame copy */
!v4lconvert_supported_dst_format(dest_fmt->fmt.pix.pixelformat)) {
int to_copy = MIN(dest_size, src_size);
//this changes what is happening to src, so I could impact the raw data
/*int i =0;
uint8_t *tmp=src;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}*/
printf(“No conversion needed!!\n”);//this means that returns the copy!
memcpy(dest, src, to_copy);
//this changes what is happening to dest
/*int i =0;
uint8_t *tmp=dest;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}*/
return to_copy;
}
printf(“Should see this print out\n\n\n”);

/* sanity check, is the dest buffer large enough? */
switch (my_dest_fmt.fmt.pix.pixelformat) {
case V4L2_PIX_FMT_RGB24:
case V4L2_PIX_FMT_BGR24:
dest_needed = my_dest_fmt.fmt.pix.width * my_dest_fmt.fmt.pix.height * 3;
temp_needed = my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3;
break;
case V4L2_PIX_FMT_YUV420:
case V4L2_PIX_FMT_YVU420:
dest_needed =
my_dest_fmt.fmt.pix.width * my_dest_fmt.fmt.pix.height * 3 / 2;
temp_needed =
my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3 / 2;
break;
default:
V4LCONVERT_ERR(“Unknown dest format in conversion\n”);
errno = EINVAL;
return -1;
}

if (dest_size < dest_needed) {
V4LCONVERT_ERR(“destination buffer too small (%d < %d)\n”,
dest_size, dest_needed);
errno = EFAULT;
return -1;
}
/* Sometimes we need foo -> rgb -> bar as video processing (whitebalance,
etc.) can only be done on rgb data */
if (processing && v4lconvert_processing_needs_double_conversion(
my_src_fmt.fmt.pix.pixelformat,
my_dest_fmt.fmt.pix.pixelformat))
convert = 2;
else if (my_dest_fmt.fmt.pix.pixelformat !=
my_src_fmt.fmt.pix.pixelformat ||
/* Special case if we do not need to do conversion, but we
are not doing any other step involving copying either,
force going through convert_pixfmt to copy the data from
source to dest */
(!rotate90 && !hflip && !vflip && !crop))
convert = 1;

/* convert_pixfmt (only if convert == 2) -> processing -> convert_pixfmt ->
rotate -> flip -> crop, all steps are optional */
if (convert == 2) {
convert1_dest = v4lconvert_alloc_buffer(
my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3,
&data->convert1_buf, &data->convert1_buf_size);
if (!convert1_dest)
return v4lconvert_oom_error(data);

convert1_dest_size =
my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3;
convert2_src = convert1_dest;
}

if (convert && (rotate90 || hflip || vflip || crop)) {
convert2_dest = v4lconvert_alloc_buffer(temp_needed,
&data->convert2_buf, &data->convert2_buf_size);
if (!convert2_dest)
return v4lconvert_oom_error(data);

convert2_dest_size = temp_needed;
rotate90_src = flip_src = crop_src = convert2_dest;
}

if (rotate90 && (hflip || vflip || crop)) {
rotate90_dest = v4lconvert_alloc_buffer(temp_needed,
&data->rotate90_buf, &data->rotate90_buf_size);
if (!rotate90_dest)
return v4lconvert_oom_error(data);

flip_src = crop_src = rotate90_dest;
}

if ((vflip || hflip) && crop) {
flip_dest = v4lconvert_alloc_buffer(temp_needed, &data->flip_buf,
&data->flip_buf_size);
if (!flip_dest)
return v4lconvert_oom_error(data);

crop_src = flip_dest;
}

/* Done setting sources / dest and allocating intermediate buffers,
real conversion / processing / … starts here. */
if (convert == 2) {
res = v4lconvert_convert_pixfmt(data, src, src_size,
convert1_dest, convert1_dest_size,
&my_src_fmt,
V4L2_PIX_FMT_RGB24);
if (res)
return res;

src_size = my_src_fmt.fmt.pix.sizeimage;
}

if (processing)
v4lprocessing_processing(data->processing, convert2_src, &my_src_fmt);

if (convert) {
res = v4lconvert_convert_pixfmt(data, convert2_src, src_size,
convert2_dest, convert2_dest_size,
&my_src_fmt,
my_dest_fmt.fmt.pix.pixelformat);
if (res)
return res;

src_size = my_src_fmt.fmt.pix.sizeimage;

/* We call processing here again in case the source format was not
rgb, but the dest is. v4lprocessing checks it self it only actually
does the processing once per frame. */
if (processing)
v4lprocessing_processing(data->processing, convert2_dest, &my_src_fmt);
}

if (rotate90)
v4lconvert_rotate90(rotate90_src, rotate90_dest, &my_src_fmt);

if (hflip || vflip)
v4lconvert_flip(flip_src, flip_dest, &my_src_fmt, hflip, vflip);

if (crop) {
v4lconvert_crop(crop_src, dest, &my_src_fmt, &my_dest_fmt);}

printf(“end of v4lconvert_convert\n”);

return dest_needed;
}

I wanted to see how early far up the function am I able to change the source. So I added the bold and underlined part to the program and was able to impact the final image. This let’s me know that the data has been written before that function.

I then went back to libv4l2.c and looked into the following piece of code (my changes are in bold):

static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
unsigned char *dest, int dest_size)
{
const int max_tries = V4L2_IGNORE_FIRST_FRAME_ERRORS + 1;
int result, tries = max_tries, frame_info_gen;

/* Make sure we have the real v4l2 buffers mapped */
result = v4l2_map_buffers(index);
printf(“inside v4l2_dequeue_and_convert\n”);
if (result)
return result;

do {
frame_info_gen = devices[index].frame_info_generation;
pthread_mutex_unlock(&devices[index].stream_lock);
result = devices[index].dev_ops->ioctl(
devices[index].dev_ops_priv,
devices[index].fd, VIDIOC_DQBUF, buf);
pthread_mutex_lock(&devices[index].stream_lock);
if (result) {
if (errno != EAGAIN) {
int saved_err = errno;

V4L2_PERROR(“dequeuing buf”);
errno = saved_err;
}
return result;
}

devices[index].frame_queued &= ~(1 << buf->index);

if (frame_info_gen != devices[index].frame_info_generation) {
errno = -EINVAL;
return -1;
}
//special_func(dest,30000);//this doesn’t change the destination
special_func(devices[index].frame_pointers[buf->index],30000);//this changes the src

result = v4lconvert_convert(devices[index].convert,
&devices[index].src_fmt, &devices[index].dest_fmt,
devices[index].frame_pointers[buf->index],
buf->bytesused, dest ? dest : (devices[index].convert_mmap_buf +
buf->index * devices[index].convert_mmap_frame_size),
dest_size);
//special_func(dest,30000);//this changes the video

if (devices[index].first_frame) {
/* Always treat convert errors as EAGAIN during the first few frames, as
some cams produce bad frames at the start of the stream
(hsync and vsync still syncing ??). */
if (result < 0)
errno = EAGAIN;
devices[index].first_frame–;
}

if (result < 0) {
int saved_err = errno;

if (errno == EAGAIN || errno == EPIPE)
V4L2_LOG(“warning error while converting frame data: %s”,
v4lconvert_get_error_message(devices[index].convert));
else
V4L2_LOG_ERR(“converting / decoding frame data: %s”,
v4lconvert_get_error_message(devices[index].convert));

/*
* If this is the last try, and the frame is short
* we will return the (short) buffer to the caller,
* so we must not re-queue it then!
*/
if (!(tries == 1 && errno == EPIPE))
v4l2_queue_read_buffer(index, buf->index);
errno = saved_err;
}
tries–;
} while (result < 0 && (errno == EAGAIN || errno == EPIPE) && tries);

if (result < 0 && errno == EAGAIN) {
V4L2_LOG_ERR(“got %d consecutive frame decode errors, last error: %s”,
max_tries, v4lconvert_get_error_message(devices[index].convert));
errno = EIO;
}

if (result < 0 && errno == EPIPE) {
V4L2_LOG(“got %d consecutive short frame errors, ”
“returning short frame”, max_tries);
result = devices[index].dest_fmt.fmt.pix.sizeimage;
errno = 0;
}
//special_func(dest,30000);
/*int i =0;
uint8_t *tmp=dest;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}*/

return result;
}

The underlined and bolded line is showing me that I am able to change the src pointer in the device. This results in the following image:

Screenshot from 2019-05-13 21-48-23.png

I was able to change the video because it came out green. So that means

devices[index].frame_pointers[buf->index]

stores the pointer to where the data is written to. Let’s see where this leads us to.

 

 

Trying to understand UVC Part 2

Full disclosure: I am just trying to track guvcview all the way to usb (I don’t know if this possible or how likely it is to work). I may make many mistakes. This is my way of recording my work and hopefully it can help other people.

Once I got into libv4l2.c which is in ~/v4l-utils/lib/libv4l2/libv4l2.c

I then went on to search for v4l2_read and here is what I saw (my additions are in bold):

ssize_t v4l2_read(int fd, void *dest, size_t n)
{
ssize_t result;
int saved_errno;
int index = v4l2_get_index(fd);

if (index == -1)
return SYS_READ(fd, dest, n);

if (!devices[index].dev_ops->read) {
errno = EINVAL;
return -1;
}
printf(“INSiDE V4L2_READ! -horrible\n”);

pthread_mutex_lock(&devices[index].stream_lock);

/* When not converting and the device supports read(), let the kernel handle
it */
if (devices[index].convert == NULL ||
((devices[index].flags & V4L2_SUPPORTS_READ) &&
!v4l2_needs_conversion(index))) {
result = devices[index].dev_ops->read(
devices[index].dev_ops_priv,
fd, dest, n);
printf(“In 1st if statement\n”);
goto leave;
}

/* Since we need to do conversion try to use mmap (streaming) mode under
the hood as that safes a memcpy for each frame read.

Note sometimes this will fail as some drivers (at least gspca) do not allow
switching from read mode to mmap mode and they assume read() mode if a
select or poll() is done before any buffers are requested. So using mmap
mode under the hood will fail if a select() or poll() is done before the
first emulated read() call. */
if (!(devices[index].flags & V4L2_STREAM_CONTROLLED_BY_READ) &&
!(devices[index].flags & V4L2_USE_READ_FOR_READ)) {
result = v4l2_activate_read_stream(index);
printf(“In 2nd if statement\n”);
if (result) {
/* Activating mmap mode failed, use read() instead */
devices[index].flags |= V4L2_USE_READ_FOR_READ;
/* The read call done by v4l2_read_and_convert will start the stream */
devices[index].first_frame = V4L2_IGNORE_FIRST_FRAME_ERRORS;
}
}

if (devices[index].flags & V4L2_USE_READ_FOR_READ) {
result = v4l2_read_and_convert(index, dest, n);
printf(“In 3rd if statement \n”);
} else {
printf(“In else statement\n”);
struct v4l2_buffer buf;

buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
result = v4l2_dequeue_and_convert(index, &buf, dest, n);
printf(“bytes_used (result) %u\n”,result);
printf(“buf->sequence %u\n”,buf.sequence);
printf(“buf->memory %u\n”,buf.memory);
printf(“index %d\n”, index);
printf(“n: %d\n”,n);

if (result >= 0) {
printf(“The if statement inside the else\n”);
v4l2_queue_read_buffer(index, buf.index);
}
}

leave:
saved_errno = errno;
pthread_mutex_unlock(&devices[index].stream_lock);
errno = saved_errno;

return result;
}

I placed a bunch of print statements to see what was being printed out (since I am not too great at C I have to use very simple programs to see what I am doing).

Here was part of guvcview output (interesting parts are underlined):

AUDIO: Portaudio device changed to 4
V4L2_CORE: checking format: YUYV
V4L2_CORE: allocating frame buffers
V4L2_CORE: trying to change fps to 1/25
GUVCVIEW: created capture thread with tid: 1924413184
GUVCVIEW: capture thread (tid: 23593)
RENDER: Initializing SDL2 render
RENDER: video display 0 -> 1920x1080px @ 60hz
RENDER: setting window size to 640×480

RENDER: Available SDL2 rendering drivers:
0: opengl
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [ ]
SDL_RENDERER_ACCELERATED [X]
SDL_RENDERER_PRESENTVSYNC [X]
1: opengles2
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [ ]
SDL_RENDERER_ACCELERATED [X]
SDL_RENDERER_PRESENTVSYNC [X]
2: software
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [X]
SDL_RENDERER_ACCELERATED [ ]
SDL_RENDERER_PRESENTVSYNC [ ]
RENDER: rendering driver in use: opengl
SDL_RENDERER_TARGETTEXTURE [X]
SDL_RENDERER_SOFTWARE [ ]
SDL_RENDERER_ACCELERATED [X]
SDL_RENDERER_PRESENTVSYNC [X]
V4L2_CORE: (VIDIOC_STREAMON) stream_status = STRM_OK
GUVCVIEW: joining capture thread
vd->mutex 0x5637c0c6be10
IO_READ
INSiDE V4L2_READ! -horrible
In 2nd if statement
In else statement
bytes_used (result) 4294967295
buf->sequence 0
buf->memory 1
index 0
n: 921600
vd->mutex 0x5637c0c6be10
IO_READ
INSiDE V4L2_READ! -horrible
In else statement
bytes_used (result) 614400
buf->sequence 0
buf->memory 1
index 0
n: 921600
The if statement inside the else
V4L2_CORE: process frame queue index 0
V4L2CORE: (fps) ref:0 ts:459274830386622 frames:1
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7f148dca9010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x5637c0c6be10
IO_READ
INSiDE V4L2_READ! -horrible
In else statement
bytes_used (result) 614400
buf->sequence 1
buf->memory 1
index 0
n: 921600
The if statement inside the else
V4L2_CORE: process frame queue index 0
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7f148dca9010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
vd->mutex 0x5637c0c6be10
IO_READ
INSiDE V4L2_READ! -horrible
In else statement
bytes_used (result) 614400
buf->sequence 2
buf->memory 1
index 0
n: 921600

It keeps going in the else statement and the if statement inside the else . Things are going all over the map so then I decided to dig deeper in the code. I thought the following line might be useful:

struct v4l2_buffer buf;

I then tried looking online for information about what that was because it didn’t seem to do anything with the raw data and here is some information that I found (link: http://www.hep.by/gnu/kernel/media/buffer.html):

A buffer contains data exchanged by application and driver using one of the Streaming I/O methods. In the multi-planar API, the data is held in planes, while the buffer structure acts as a container for the planes. Only pointers to buffers (planes) are exchanged, the data itself is not copied. These pointers, together with meta-information like timestamps or field parity, are stored in a struct v4l2_buffer, argument to the VIDIOC_QUERYBUF, VIDIOC_QBUF and VIDIOC_DQBUF ioctl. In the multi-planar API, some plane-specific members of struct v4l2_buffer, such as pointers and sizes for each plane, are stored in struct v4l2_plane instead. In that case, struct v4l2_buffer contains an array of plane structures”

The bolded and underlined part was very interesting because it says that the v4l2_buffer only contains the information about the data but not the raw data!!! They are also not copying data but telling the computer where the data is stored. Ok with that understanding it seems that the v4l2_buffer tells us where the data is and the actual raw data is it planes? I guess time to keep going.

I then proceded to look v4l2_read function i libv4l2.c and started to see the v4l2_read function (my edits in bold):

vim ~/v4l-utils/lib/libv4l2/libv4l2.c

vim ~/v4l-utils/lib/libv4l2/libv4l2.c

ssize_t v4l2_read(int fd, void *dest, size_t n)
{
ssize_t result;
int saved_errno;
int index = v4l2_get_index(fd);

if (index == -1)
return SYS_READ(fd, dest, n);

if (!devices[index].dev_ops->read) {
errno = EINVAL;
return -1;
}
printf(“INSiDE V4L2_READ! -horrible\n”);

pthread_mutex_lock(&devices[index].stream_lock);

/* When not converting and the device supports read(), let the kernel handle
it */
if (devices[index].convert == NULL ||
((devices[index].flags & V4L2_SUPPORTS_READ) &&
!v4l2_needs_conversion(index))) {
result = devices[index].dev_ops->read(
devices[index].dev_ops_priv,
fd, dest, n);
printf(“In 1st if statement\n”);
goto leave;
}

/* Since we need to do conversion try to use mmap (streaming) mode under
the hood as that safes a memcpy for each frame read.

Note sometimes this will fail as some drivers (at least gspca) do not allow
switching from read mode to mmap mode and they assume read() mode if a
select or poll() is done before any buffers are requested. So using mmap
mode under the hood will fail if a select() or poll() is done before the
first emulated read() call. */
if (!(devices[index].flags & V4L2_STREAM_CONTROLLED_BY_READ) &&
!(devices[index].flags & V4L2_USE_READ_FOR_READ)) {
result = v4l2_activate_read_stream(index);
printf(“In 2nd if statement\n”);
if (result) {
/* Activating mmap mode failed, use read() instead */
devices[index].flags |= V4L2_USE_READ_FOR_READ;
/* The read call done by v4l2_read_and_convert will start the stream */
devices[index].first_frame = V4L2_IGNORE_FIRST_FRAME_ERRORS;
}
}

if (devices[index].flags & V4L2_USE_READ_FOR_READ) {
result = v4l2_read_and_convert(index, dest, n);
printf(“In 3rd if statement \n”);
} else {
printf(“In else statement with 3rd if\n”);
struct v4l2_buffer buf;

buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
result = v4l2_dequeue_and_convert(index, &buf, dest, n);
printf(“bytes_used (result) %lu\n”,result);
printf(“buf->sequence 0x%x\n”,buf.sequence);
printf(“buf->memory 0x%x\n”,buf.memory);
printf(“buf->offset 0x%x\n”,buf.m.offset);
printf(“index %d\n”, index);
printf(“n: %lx\n”,n);
printf(“buf->v4l2_plane %p\n”,buf.m.planes);
//I can’t get anything to happen with the below if statment. It seems as if
//buf.m.planes has nothing populated and I keep getting errors
/*if((buf.m.planes!=NULL) && (buf.m.planes->bytesused>0)) {
printf(“inside my own if statement\n”);
printf(“buf.m.planes->bytesused %i\n”,buf.m.planes->bytesused);
//printf(“buf.m.planes->m.mem_offset 0x%x\n”,buf.m.planes->m.mem_offset);
//printf(“buf.m.planes->m.userptr 0x%lx\n”,buf.m.planes->m.userptr);
//printf(“buf.m.planes->data_offset 0x%x\n”,buf.m.planes->data_offset);
}*/
printf(“buf->userptr %lx\n”,buf.m.userptr);
printf(“buf.m.length %i\n”,buf.length);
printf(“buf.type %x\n”,buf.type);
printf(“buf.flags %x\n”,buf.flags);
//this seems to be giving a segmentation fault
/*if(buf.m.userptr) {
if((result<1000000) && (result>0)){
int i =0;
uint8_t *tmp=(uint8_t *)(buf.memory);
for(i=0;i<2;i++) {
*tmp=0;
tmp++;
}
}
}*/

//printf(“dest->memory %u\n”,dest.memory);

if (result >= 0) {
printf(“The if statement inside the else\n”);
v4l2_queue_read_buffer(index, buf.index);

printf(“devices[index].frame_pointers[0] %02x\n”,(unsigned int)devices[index].frame_pointers[0]);
}
}

leave:
saved_errno = errno;
pthread_mutex_unlock(&devices[index].stream_lock);
errno = saved_errno;
if(result > 0) {
int i =0;
uint8_t *tmp=(uint8_t *)dest;
printf(“dest address: %p\n”,dest);
printf(“v4l2_read frame: “);
for(i=0;i<3000;i++) {
//printf(“%d,”,*tmp);
*tmp=0;
tmp++;
}
printf(“\n”);
}

return result;
}

So you can see all the print commands I had! It took me a while to figure out what the heck to do with it all! If you look at the underlined part I kept trying to access the v4l2_planes ptr but kept getting a segmentation fault as soon as the video would start. I would get the right size but could never access any of the data in v4l2_planes. I realized by looking back at v4l2_core.c and saw that

vd->buf.bytesused = v4l2_read (vd->fd, vd->mem[vd->buf.index], vd->buf.length);

I thought maybe the raw frame and the vd->mem are related so I tried the following command in v4l2_core.c

v4l2_frame_buff_t *v4l2core_get_frame(v4l2_dev_t *vd)
{
/*asserts*/
assert(vd != NULL);

/*for H264 streams request a IDR frame with SPS and PPS data if it’s the first frame*/
if(vd->requested_fmt == V4L2_PIX_FMT_H264 && vd->frame_index < 1)
request_h264_frame_type(vd, PICTURE_TYPE_IDR_FULL);

int res = 0;
int ret = check_frame_available(vd);

int qind = -1;

if (ret < 0)
return NULL;

int bytes_used = 0;
printf(“vd->mutex %p\n”,__PMUTEX);

switch(vd->cap_meth)
{
case IO_READ:

/*lock the mutex*/
printf(“IO_READ\n”);
__LOCK_MUTEX( __PMUTEX );
if(vd->streaming == STRM_OK)
{
vd->buf.bytesused = v4l2_read (vd->fd, vd->mem[vd->buf.index], vd->buf.length);
bytes_used = vd->buf.bytesused;

if(bytes_used > 0)
qind = process_input_buffer(vd);
}
else res = -1;
/*unlock the mutex*/
__UNLOCK_MUTEX( __PMUTEX );
if (-1==bytes_used)
return NULL;
int i =0;
//uint8_t *tmp=vd->frame_queue[qind].raw_frame;
uint8_t *tmp = vd->mem[vd->buf.index];
printf(“frame data: “);
for(i=0;i<30;i++) {
printf(“%d,”,*tmp);
tmp++;
}
printf(“\n”);

if(res < 0)
return NULL;

then I also made changes to

static int process_input_buffer(v4l2_dev_t *vd)
{
/*get next available frame in queue*/
int qind = get_next_ready_frame(vd);

if(verbosity > 2)
printf(“V4L2_CORE: process frame queue index %i\n”, qind);

if(qind < 0 || qind >= vd->frame_queue_size)
{
if(verbosity > 2)
fprintf(stderr,”V4L2_CORE: frame queue index %i is invalid (no free frames in queue?)\n”, qind);
return -1;
}

vd->frame_queue[qind].status = FRAME_DECODING;

/*
* driver timestamp is unreliable
* use monotonic system time
*/
vd->frame_queue[qind].timestamp = ns_time_monotonic();

vd->frame_queue[qind].index = vd->buf.index;

vd->frame_index++;

vd->frame_queue[qind].raw_frame_size = vd->buf.bytesused;
if(vd->frame_queue[qind].raw_frame_size == 0)
{
if(verbosity > 1)
fprintf(stderr, “V4L2_CORE: VIDIOC_QBUF returned buf.bytesused = 0 \n”);
}

/*point vd->raw_frame to current frame buffer*/
vd->frame_queue[qind].raw_frame = vd->mem[vd->buf.index];
int i = 0;
uint8_t *tmp=vd->frame_queue[qind].raw_frame;
//uint8_t *tmp=vd->mem[vd->buf.index];
printf(“raw data location: %p\n”,vd->mem[vd->buf.index]);
printf(“process_queue_frame (vd->mem) frame: “);
for(i=0;i<30;i++) {
printf(“%d,”,*tmp);
tmp++;
}
printf(“\n”);

/*determine real fps every 3 sec aprox.*/
fps_frame_count++;

if(vd->frame_queue[qind].timestamp – fps_ref_ts >= (3 * NSEC_PER_SEC))
{
if(verbosity > 2)
printf(“V4L2CORE: (fps) ref:%”PRId64″ ts:%”PRId64″ frames:%i\n”,
fps_ref_ts, vd->frame_queue[qind].timestamp, fps_frame_count);
vd->real_fps = (double) (fps_frame_count * NSEC_PER_SEC) / (double) (vd->frame_queue[qind].timestamp – fps_ref_ts);
fps_frame_count = 0;
fps_ref_ts = vd->frame_queue[qind].timestamp;
}

return qind;
}

This was comparing data from vd-mem and the raw frame and the output was:

buf->userptr 558000096000
buf.m.length 614400
buf.type 1
buf.flags 12001
The if statement inside the else
devices[index].frame_pointers[0] 1f875000
V4L2_CORE: process frame queue index 0
raw data location: 0x7f593a57a010
process_queue_frame (vd->mem) frame: 146,134,150,124,146,132,141,127,142,133,145,127,151,134,152,126,146,134,142,124,141,132,138,127,128,135,124,125,123,135,
frame data: 146,134,150,124,146,132,141,127,142,133,145,127,151,134,152,126,146,134,142,124,141,132,138,127,128,135,124,125,123,135,
V4L2_CORE: decoding raw frame of size 614400 at 0x0x7f593a57a010
the format is 1448695129
inside yuyv!!
NON-BAYER DATA!
RENDER: event 0 -> callback 0
GUVCVIEW: free profile name
GUVCVIEW: free profile path
GUVCVIEW: free video name
GUVCVIEW: free video path
GUVCVIEW: free photo name
GUVCVIEW: free photo path
GUVCVIEW: close GUI API
V4L2_CORE: (VIDIOC_STREAMOFF) stream_status = S

As you can see the vd->mem and the raw share same values! So this leads me to believe that vd-mem and raw_frame are related.

If that is true I should be able to directly edit the dest input argument in v4l2_read.

I did it the following change to v4l2_read(shown in bold):

printf(“buf->userptr %lx\n”,buf.m.userptr);
printf(“buf.m.length %i\n”,buf.length);
printf(“buf.type %x\n”,buf.type);
printf(“buf.flags %x\n”,buf.flags);
//this seems to be giving a segmentation fault
/*if(buf.m.userptr) {
if((result<1000000) && (result>0)){
int i =0;
uint8_t *tmp=(uint8_t *)(buf.memory);
for(i=0;i<2;i++) {
*tmp=0;
tmp++;
}
}
}*/

//printf(“dest->memory %u\n”,dest.memory);

if (result >= 0) {
printf(“The if statement inside the else\n”);
v4l2_queue_read_buffer(index, buf.index);

printf(“devices[index].frame_pointers[0] %02x\n”,(unsigned int)devices[index].frame_pointers[0]);
}
}

leave:
saved_errno = errno;
pthread_mutex_unlock(&devices[index].stream_lock);
errno = saved_errno;
if(result > 0) {
int i =0;
uint8_t *tmp=(uint8_t *)dest;
printf(“dest address: %p\n”,dest);
printf(“v4l2_read frame: “);
for(i=0;i<3000;i++) {
//printf(“%d,”,*tmp);
*tmp=0;
tmp++;
}
printf(“\n”);
}

return result;
}

 

So that lead to Screenshot from 2019-05-12 17-39-45

So if you look carefully you can see the top part of the image is green! That means I was directly able to manipulate the image. Now I have gotten into the v4l-utils folder, so I hope this is the right direction.

If my understanding is correct it seems we are allocating space for the buffers to write data to.

I spent a long time chasing where to find the raw data dn I kept getting lost, so finally I start looking closer at this piece of code:

if (devices[index].flags & V4L2_USE_READ_FOR_READ) {
result = v4l2_read_and_convert(index, dest, n);
printf(“In 3rd if statement \n”);
} else {
printf(“In else statement with 3rd if\n”);
struct v4l2_buffer buf;

buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
result = v4l2_dequeue_and_convert(index, &buf, dest, n);
printf(“buf addres %p\n”,&buf);

The function in “v4l2_dequeue_and_convert” and the code looked like this:

void special_func(uint8_t *mem_oi,int val) {
int i =0;
uint8_t *tmp = mem_oi;
for(i=0;i<val;i++) {
*tmp=0;
tmp++;
}
}

static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
unsigned char *dest, int dest_size)
{
const int max_tries = V4L2_IGNORE_FIRST_FRAME_ERRORS + 1;
int result, tries = max_tries, frame_info_gen;

/* Make sure we have the real v4l2 buffers mapped */
result = v4l2_map_buffers(index);
printf(“inside v4l2_dequeue_and_convert\n”);
if (result)
return result;

do {
frame_info_gen = devices[index].frame_info_generation;
pthread_mutex_unlock(&devices[index].stream_lock);
result = devices[index].dev_ops->ioctl(
devices[index].dev_ops_priv,
devices[index].fd, VIDIOC_DQBUF, buf);
pthread_mutex_lock(&devices[index].stream_lock);
if (result) {
if (errno != EAGAIN) {
int saved_err = errno;

V4L2_PERROR(“dequeuing buf”);
errno = saved_err;
}
return result;
}

devices[index].frame_queued &= ~(1 << buf->index);

if (frame_info_gen != devices[index].frame_info_generation) {
errno = -EINVAL;
return -1;
}
//special_func(dest,30000);//this doesn’t change the destination

result = v4lconvert_convert(devices[index].convert,
&devices[index].src_fmt, &devices[index].dest_fmt,
devices[index].frame_pointers[buf->index],
buf->bytesused, dest ? dest : (devices[index].convert_mmap_buf +
buf->index * devices[index].convert_mmap_frame_size),
dest_size);
//special_func(dest,30000);//this changes the video

if (devices[index].first_frame) {
/* Always treat convert errors as EAGAIN during the first few frames, as
some cams produce bad frames at the start of the stream
(hsync and vsync still syncing ??). */
if (result < 0)
errno = EAGAIN;
devices[index].first_frame–;
}

if (result < 0) {
int saved_err = errno;

if (errno == EAGAIN || errno == EPIPE)
V4L2_LOG(“warning error while converting frame data: %s”,
v4lconvert_get_error_message(devices[index].convert));
else
V4L2_LOG_ERR(“converting / decoding frame data: %s”,
v4lconvert_get_error_message(devices[index].convert));

/*
* If this is the last try, and the frame is short
* we will return the (short) buffer to the caller,
* so we must not re-queue it then!
*/
if (!(tries == 1 && errno == EPIPE))
v4l2_queue_read_buffer(index, buf->index);
errno = saved_err;
}
tries–;
} while (result < 0 && (errno == EAGAIN || errno == EPIPE) && tries);

if (result < 0 && errno == EAGAIN) {
V4L2_LOG_ERR(“got %d consecutive frame decode errors, last error: %s”,
max_tries, v4lconvert_get_error_message(devices[index].convert));
errno = EIO;
}

if (result < 0 && errno == EPIPE) {
V4L2_LOG(“got %d consecutive short frame errors, ”
“returning short frame”, max_tries);
result = devices[index].dest_fmt.fmt.pix.sizeimage;
errno = 0;
}
//special_func(dest,30000); //this changes the video
/*int i =0;
uint8_t *tmp=dest;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}*/

return result;
}

I went into where my special function was changing the video which led me to v4lconvert_convert (showed in underline in the code above)

So I tried to find out where that function was actually being used and here is my grep command:

xxx:~$ grep -r –include “*.c” “v4lconvert_convert(”
grep: .dbus: Permission denied
v4l-utils/lib/libv4lconvert/libv4lconvert.c:int v4lconvert_convert(struct v4lconvert_data *data,
v4l-utils/lib/libv4l2/libv4l2.c: result = v4lconvert_convert(devices[index].convert,
v4l-utils/lib/libv4l2/libv4l2.c: result = v4lconvert_convert(devices[index].convert,

As you can see that this function is defined in libv4lconvert.c here is the function (my changes in bold):

int v4lconvert_convert(struct v4lconvert_data *data,
const struct v4l2_format *src_fmt, /* in */
const struct v4l2_format *dest_fmt, /* in */
unsigned char *src, int src_size, unsigned char *dest, int dest_size)
{
printf(“v4lconvert_convert\n”);
int res, dest_needed, temp_needed, processing, convert = 0;
int rotate90, vflip, hflip, crop;
unsigned char *convert1_dest = dest;
int convert1_dest_size = dest_size;
unsigned char *convert2_src = src, *convert2_dest = dest;
int convert2_dest_size = dest_size;
unsigned char *rotate90_src = src, *rotate90_dest = dest;
unsigned char *flip_src = src, *flip_dest = dest;
unsigned char *crop_src = src;
struct v4l2_format my_src_fmt = *src_fmt;
struct v4l2_format my_dest_fmt = *dest_fmt;

processing = v4lprocessing_pre_processing(data->processing);
rotate90 = data->control_flags & V4LCONTROL_ROTATED_90_JPEG;
hflip = v4lcontrol_get_ctrl(data->control, V4LCONTROL_HFLIP);
vflip = v4lcontrol_get_ctrl(data->control, V4LCONTROL_VFLIP);
crop = my_dest_fmt.fmt.pix.width != my_src_fmt.fmt.pix.width ||
my_dest_fmt.fmt.pix.height != my_src_fmt.fmt.pix.height;

if (/* If no conversion/processing is needed */
(src_fmt->fmt.pix.pixelformat == dest_fmt->fmt.pix.pixelformat &&
!processing && !rotate90 && !hflip && !vflip && !crop) ||
/* or if we should do processing/rotating/flipping but the app tries to
use the native cam format, we just return an unprocessed frame copy */
!v4lconvert_supported_dst_format(dest_fmt->fmt.pix.pixelformat)) {
int to_copy = MIN(dest_size, src_size);
//this changes what is happening to src, so I could impact the raw data
int i =0;
uint8_t *tmp=src;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}
printf(“No conversion needed!!\n”);//this means that returns the copy!
memcpy(dest, src, to_copy);
//this changes what is happening to dest
/*int i =0;
uint8_t *tmp=dest;
for(i=0;i<30000;i++) {
*tmp=0;
tmp++;
}*/
return to_copy;
}

/* sanity check, is the dest buffer large enough? */
switch (my_dest_fmt.fmt.pix.pixelformat) {
case V4L2_PIX_FMT_RGB24:
case V4L2_PIX_FMT_BGR24:
dest_needed = my_dest_fmt.fmt.pix.width * my_dest_fmt.fmt.pix.height * 3;
temp_needed = my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3;
break;
case V4L2_PIX_FMT_YUV420:
case V4L2_PIX_FMT_YVU420:
dest_needed =
my_dest_fmt.fmt.pix.width * my_dest_fmt.fmt.pix.height * 3 / 2;
temp_needed =
my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3 / 2;
break;
default:
V4LCONVERT_ERR(“Unknown dest format in conversion\n”);
errno = EINVAL;
return -1;
}

if (dest_size < dest_needed) {
V4LCONVERT_ERR(“destination buffer too small (%d < %d)\n”,
dest_size, dest_needed);
errno = EFAULT;
return -1;
}
/* Sometimes we need foo -> rgb -> bar as video processing (whitebalance,
etc.) can only be done on rgb data */
if (processing && v4lconvert_processing_needs_double_conversion(
my_src_fmt.fmt.pix.pixelformat,
my_dest_fmt.fmt.pix.pixelformat))
convert = 2;
else if (my_dest_fmt.fmt.pix.pixelformat !=
my_src_fmt.fmt.pix.pixelformat ||
/* Special case if we do not need to do conversion, but we
are not doing any other step involving copying either,
force going through convert_pixfmt to copy the data from
source to dest */
(!rotate90 && !hflip && !vflip && !crop))
convert = 1;

/* convert_pixfmt (only if convert == 2) -> processing -> convert_pixfmt ->
rotate -> flip -> crop, all steps are optional */
if (convert == 2) {
convert1_dest = v4lconvert_alloc_buffer(
my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3,
&data->convert1_buf, &data->convert1_buf_size);
if (!convert1_dest)
return v4lconvert_oom_error(data);

convert1_dest_size =
my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3;
convert2_src = convert1_dest;
}

if (convert && (rotate90 || hflip || vflip || crop)) {
convert2_dest = v4lconvert_alloc_buffer(temp_needed,
&data->convert2_buf, &data->convert2_buf_size);
if (!convert2_dest)
return v4lconvert_oom_error(data);

convert2_dest_size = temp_needed;
rotate90_src = flip_src = crop_src = convert2_dest;
}

if (rotate90 && (hflip || vflip || crop)) {
rotate90_dest = v4lconvert_alloc_buffer(temp_needed,
&data->rotate90_buf, &data->rotate90_buf_size);
if (!rotate90_dest)
return v4lconvert_oom_error(data);

flip_src = crop_src = rotate90_dest;
}

if ((vflip || hflip) && crop) {
flip_dest = v4lconvert_alloc_buffer(temp_needed, &data->flip_buf,
&data->flip_buf_size);
if (!flip_dest)
return v4lconvert_oom_error(data);

crop_src = flip_dest;
}

/* Done setting sources / dest and allocating intermediate buffers,
real conversion / processing / … starts here. */
if (convert == 2) {
res = v4lconvert_convert_pixfmt(data, src, src_size,
convert1_dest, convert1_dest_size,
&my_src_fmt,
V4L2_PIX_FMT_RGB24);
if (res)
return res;

src_size = my_src_fmt.fmt.pix.sizeimage;
}

if (processing)
v4lprocessing_processing(data->processing, convert2_src, &my_src_fmt);

if (convert) {
res = v4lconvert_convert_pixfmt(data, convert2_src, src_size,
convert2_dest, convert2_dest_size,
&my_src_fmt,
my_dest_fmt.fmt.pix.pixelformat);
if (res)
return res;

src_size = my_src_fmt.fmt.pix.sizeimage;

/* We call processing here again in case the source format was not
rgb, but the dest is. v4lprocessing checks it self it only actually
does the processing once per frame. */
if (processing)
v4lprocessing_processing(data->processing, convert2_dest, &my_src_fmt);
}

if (rotate90)
v4lconvert_rotate90(rotate90_src, rotate90_dest, &my_src_fmt);

if (hflip || vflip)
v4lconvert_flip(flip_src, flip_dest, &my_src_fmt, hflip, vflip);

if (crop) {
v4lconvert_crop(crop_src, dest, &my_src_fmt, &my_dest_fmt);}

printf(“end of v4lconvert_convert\n”);

return dest_needed;
}

I was able to change the dest data and the src data.

I did not see printf(“end of v4lconvert_convert\n”); so I assumed something was happening before that. I saw that there was something that didnt’ require any processing. Here is what the screenshot looks like:

Screenshot from 2019-05-12 22-02-45.png

(ignore the movie in the background)

After I was able to change the src I saw the following image:

Screenshot from 2019-05-12 21-42-43.png

(ignore the movie)

I have gotten step closer. Next to dig more into

That’s all for today, will keep updating with more:

Finding address of MS5607 by bit banging I2C with arduino uno in C and Sketch

This has been one my bigger solo projects to get an understanding of how bit banging works and I didn’t want to use the pre-built I2C (I squared C) module because I wanted to learn how I2C communication works. I quickly realized how difficult this actually was. In order to get a good understanding I had to first learn how I2C protocol works. I used boldinventions website to understand how the I2C protocol works. They have a great graphic of what is going on:

 

You can see that there are 2 connection one is SCL (clock) and the other is SDA (data). So the clock sets when the system reads. When the clock is high the SDA line is read. There is a master and a slave and clock is driven by the master and they both send data back and forth on the SDA line. In order to communicate with the slave we have to address the slave. So in the picture above you can see that in order to initiate the conversation we send a start signal. The start signal is sent by setting SDA to low when SCL is high. The next step is to send the address of the device and in this case they send 1011001 (0x59) where the 1s are high and 0s are low. The next bit that is sent is to either to write to that address or read from the address. The next is for the master to wait for an acknowledge to make sure the slave got the message and it responds with an ack bit where the ack bit goes low(this took me a while to get). If the ack bit stays high then it means that the message was no received(think of ack bit sending either 1 or 0, if 1 is received then the slave did not acknowledge but if 0 is received then the slave received the message). The part that is missing is sending the stop bit when the SCL is high and SDA goes from low to high. In order to get started I first started with writing my code in sketch:

int sda = 6;
int scl = 7;
void setup() {
 // put your setup code here, to run once:
 pinMode(sda,OUTPUT);
 digitalWrite(sda,HIGH);
 pinMode(scl,OUTPUT);
 digitalWrite(scl,HIGH);
 delay(3000);
 Serial.begin(9600);
 sendStop();
 sendStart();
 int counter = 0;
 int ack=1;
 String addrstr = "11101100";
 String addrstr1 = "11000010";
 String addrstr2 = "00000010";
 sendStr(addrstr1);
 Serial.println('D',HEX);
 ack = getAck();
 checkAck(counter,ack);
 /*sendStop();
 sendStart();
 ack=1;
 sendStr(addrstr2);
 Serial.println('D');
 ack = getAck();
 checkAck(counter,ack);*/
 sendStop();
 sendStart();
 ack=1;
 sendStr(addrstr);
 ack = getAck();
 checkAck(counter,ack);
 sendStop();
 Serial.println('E',HEX);
 Serial.println('E',HEX);
 Serial.println('E',HEX);
 Serial.println('E',HEX);
}

void sendReset(){
 String res = "00011110";
 int counter;
 Serial.print("Reset Data :");
 for(counter=0;counter<8;counter++){
 sendData(res[counter]-'0');
 }
 Serial.println();
}

void checkAck(int counter,int ack){
 if(ack == LOW){
 Serial.println('S',HEX);
 } 
 else{
 Serial.println('F',HEX);
 }
}

void sendStr(String addrstr){
 int counter = 0;
 for(counter=0;counter<8;counter++){
 sendData(addrstr[counter]-'0');
 }
}

void loop() {
 // put your main code here, to run repeatedly:
}

void sendZeros(int diff){
 int i = 0;
 for(i=0;i<diff;i++){
 sendData((unsigned int)('0'-'0'));
 //Serial.println("in for loop");
 }
}

unsigned int getAck(){
 Serial.println('a',HEX);
 unsigned int ack=1;
 delay(10);
 Serial.println(PIND,HEX);
 pinMode(sda,INPUT);
 delay(10);
 Serial.println(DDRD,HEX);
 Serial.println(PORTD,HEX);
 digitalWrite(scl,HEX);
 delay(10);
 Serial.println(PIND,HEX);
 ack = digitalRead(sda);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 pinMode(sda,OUTPUT);
 Serial.println(DDRD,HEX);
 Serial.println(PORTD,HEX);
 Serial.println('a',HEX);
 return ack;
}

void sendData(unsigned int data){
 Serial.println('d',HEX);
 Serial.println(data,HEX);
 //Serial.print(data,HEX);
 if(data==0){
 digitalWrite(sda,LOW);}
 else{
 digitalWrite(sda,HIGH);}
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 Serial.println('d',HEX);
}

void sendStop(){
 Serial.println('u',HEX);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(sda,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(sda,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 Serial.println('u',HEX);
}

void sendStart(){
 Serial.println('t',HEX);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(sda,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 Serial.println('t',HEX);
}

The first thing I did was find the address of the PARALLAX MS5607 which I found in the manual to be 111011Cx where Cx is the complementary value of the CSB pin which in my case was 0. I converted that to a binary string

String addrstr = "11101100";

I used the serial port a lot on this one because I needed to know what was being outputted. I created a sendStop function which

void sendStop(){
 Serial.println('u',HEX);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(sda,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(sda,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 Serial.println('u',HEX);
}
As you can see I have a lot of Serial.println because I needed to print out the exact pin states in order to see that things were being toggled properly. I then send it through the start function
void sendStart(){
 Serial.println('t',HEX);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(sda,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 Serial.println('t',HEX);
}
this allowed me to send a start signal only using the GPIO pins. Just FYI I am using pins 6 and 7 as outline above. As always I have lots of print statements because I kept running into all sorts of issues. I made a function for writing data where the bit is put is when the SCL clock goes high and had function that checked if the acknowledge worked.
unsigned int getAck(){
 Serial.println('a',HEX);
 unsigned int ack=1;
 delay(10);
 Serial.println(PIND,HEX);
 pinMode(sda,INPUT);
 delay(10);
 Serial.println(DDRD,HEX);
 Serial.println(PORTD,HEX);
 digitalWrite(scl,HEX);
 delay(10);
 Serial.println(PIND,HEX);
 ack = digitalRead(sda);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 pinMode(sda,OUTPUT);
 Serial.println(DDRD,HEX);
 Serial.println(PORTD,HEX);
 Serial.println('a',HEX);
 return ack;
}

void sendData(unsigned int data){
 Serial.println('d',HEX);
 Serial.println(data,HEX);
 //Serial.print(data,HEX);
 if(data==0){
 digitalWrite(sda,LOW);}
 else{
 digitalWrite(sda,HIGH);}
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,HIGH);
 delay(10);
 Serial.println(PIND,HEX);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.println(PIND,HEX);
 Serial.println('d',HEX);
}
What was interesting for me was to set ack==0 instead of 1 and the key part is that the slave pull SDA down which causes the SDA pin to go low. I had to output after each step because I didn’t know when things were failing and sometimes I had to output PORTD and DDRD just to make sure pins are being configured right. I sent through 2 address, where one was wrong and one is right to make sure that the wrong one doesn’t impact anything. I wanted to do this because when you are bit banging a wrong address shouldn’t do anything.
 String addrstr = "11101100";
 String addrstr1 = "11000010";
I then looked at the console output and it should be clear that when ack is 0 (slave received the message) I should get the ‘S’ character in hex otherwise I would just get ‘F’. I will show the output in a bit, but for now know that it worked. Once this was done I wanted to convert everything to C, because I wanted to :). The one thing to note is I placed a bunch of ‘E’ characters at the end of the program to let me know that when I see 4 repeating character I know that the code went through. The first try at my C program was just to mimc the ino file that I created.
filename: bit_bang_MS5607_working.c
#include<stdlib.h>
#include<stdio.h>
#include<avr/interrupt.h>
#include<avr/io.h> 
#include<util/delay.h>

#define FOSC 16000000 
#define BAUD 9600
#define MYUBRR FOSC/16/BAUD-1 
//#define MYUBRR 103
#define SCL PORTD7
#define SDA PORTD6

void USART_init(unsigned int ubrr);
void USART_transmit(char data);
unsigned char USART_recieve(void);
unsigned int getAck();
void sendStart();
void sendStop();
void checkAck(unsigned int ack);
void sendData(unsigned char data);
void sendStr(char data[8]);
void morepininfo();

int main(){
 //char data={"0000000"};//"Hello!";
 /* String addrstr = "11101100";
 String addrstr1 = "11000010";
 String addrstr2 = "00000010";
 */




char* addrstr = "11101100";
 char* addrstr1 = "11000010";
 char* addrstr2 = "10101010";
 unsigned int ack;
 USART_init(MYUBRR);
 _delay_ms(10000);
 DDRD = 0;
 PORTD = 0;
 //DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (0<<SDA)|(1<<SCL)|(1<<PORTD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 //first address
 sendStop();
 sendStart();
 sendStr(addrstr1);
 USART_transmit('D');
 ack = getAck();
 checkAck(ack);
 sendStop();
 //second address
 DDRD = 0;
 PORTD = 0;
 //DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (0<<SDA)|(1<<SCL)|(1<<PORTD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 //morepininfo();
 sendStop();
 sendStart();
 sendStr(addrstr);
 USART_transmit('D');
 ack=getAck();
 checkAck(ack);
 sendStop();
 USART_transmit('E'); 
 USART_transmit('E');
 USART_transmit('E');
 USART_transmit('E');

while(1){
 _delay_ms(10);
 PORTD |= (1<<PORTD4);
 _delay_ms(10);
 PORTD ^= (1<<PORTD4);
 _delay_ms(10);
 USART_transmit(PIND+65);
 _delay_ms(5000);
 if ((PIND)&(1<<SDA)){
 USART_transmit('i');
 _delay_ms(10);
 PORTD ^= (1<<SCL);
 }
 //USART_transmit(97);
 //data = USART_recieve();
 //USART_transmit(data);
 //_delay_ms(1000);
}
}

void morepininfo(){
 USART_transmit('p');
 USART_transmit(PORTD);
 USART_transmit(DDRD);
 USART_transmit('p');
}

void checkAck(unsigned int ack){
 if(ack == 0){
 USART_transmit('S');
 }
 else{
 USART_transmit('F');
 }
}

void sendStr(char addr[8]){
 int i = 0;
 for(i=0;i<8;i++){
 sendData(addr[i]);
 }
}

void sendStart(){
 USART_transmit('t');
 USART_transmit(PIND);
 PORTD |= (1<<SCL); //setting scl low
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (0<<SDA)|(1<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (0<<SDA)|(0<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 USART_transmit('t');
}

void sendStop(){
 USART_transmit('u');
 USART_transmit(PIND);
 PORTD = (0<<SCL)|(1<<SDA);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (0<<SDA)|(0<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (1<<SDA)|(0<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (1<<SDA)|(1<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 USART_transmit('u');
}




void sendData(unsigned char data){
 USART_transmit('d');
 int val = data-'0';
 USART_transmit(val);
 USART_transmit(PIND);
 PORTD = (val<<SDA)|(0<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (val<<SDA)|(1<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (val<<SDA)|(0<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 USART_transmit('d');
}

unsigned int getAck(){
 USART_transmit('a');
 unsigned int ack=1;
 _delay_ms(10);
 USART_transmit(PIND);
 //convert PIND6 to input
 DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4); 
 PORTD = (1<<SDA)|(1<<SCL);
 USART_transmit(DDRD);
 USART_transmit(PORTD);
 _delay_ms(10);
 PORTD |= (1<<SCL);
 //_delay_ms(10000);
 USART_transmit(PIND);
 _delay_ms(10);
 USART_transmit(PIND);
 if(~(PIND)&(1<<SDA)){
 ack = 0;}
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD |= (1<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 PORTD = (0<<SCL);
 _delay_ms(10);
 USART_transmit(PIND);
 //set pin6 to output
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 _delay_ms(10);
 USART_transmit(DDRD);
 USART_transmit(PORTD);
 USART_transmit('a');
 return ack;
}




void USART_init(unsigned int ubrr){
 UCSR0B = (1<<TXEN0)|(1<<RXEN0);

//choose asynchronous mode
 UCSR0C = 0;
 UCSR0C = (0<<UMSEL01)|(0<<UMSEL00);
 UCSR0C |= (1<<UCSZ01)|(1<<UCSZ00);
 UCSR0C |= (1<<USBS0);
 /*//setting UBRR TO 103 for 9600 baudrate
 UBRR0H = 0;
 UBRR0L = 0b01100111; */
 UBRR0H = (unsigned char)(ubrr>>8);
 UBRR0L = (unsigned char)ubrr;
}

void USART_transmit(char data){
 char temp='a';//initialize to a certain character
 //while(temp){
 //temp = *(data++);//go through each character one by one until all the data has been put out
 while( !(UCSR0A & (1<<UDRE0)) ) //
 ;
 //UDR0 = temp;
 UDR0 = data;
 //}
 _delay_ms(10);
}

unsigned char USART_recieve(void){
 /*wait for data to be recieved, this is important because the data register has to get filled up and a flag gets created when a stop bit is set high*/
 while(!(UCSR0A & (1<<RXC0) ))
 ;
 //once the stop bit is set high then read directly from UDRN
 return UDR0;
}
The UART material is the same as my previous post. So I am not going to repeat that but I used the code in order to debug my software. It is very similar to sketch file except I am directly calling the ports. I am using USART_transmit instead of Serial.println and I am setting a pin to be input by using
DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 PORTD = (1<<SDA)|(1<<SCL);
In my case PinD7 was the clock and PinD6 is the data line. Setting DDRD where bit 6 is set to 0 causes the pin to become an input pin as outline by the ATMEGA328p manual. You can see I am outputting after every single line in order to see what the pin status looks like. I was constantly comparing my sketch output with my c output.
Here is my sketch output decoded with the proper binary conversion:
intputa.csv
0x75 u
0xC1 11000001
0x43 01000011
0x3 00000011
0x43 01000011
0xC3 11000011
0x75 u
0x74 t
0xC3 11000011
0xC3 11000011
0x81 10000001
0x3 00000011
0x74 t
0x64 d
0x1 00000001
0x41 01000001
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x44 D
0x61 a
0x3 00000011
0x80 10000000
0x0 00000000
0xC3 11000011
0xC3 11000011
0xC3 11000011
0x43 01000011
0xC0 11000000
0x0 00000000
0x46 F
0x75 u
0x1 00000001
0x1 00000001
0x1 00000001
0x43 01000011
0xC3 11000011
0x75 u
0x74 t
0xC3 11000011
0xC3 11000011
0x81 10000001
0x3 00000011
0x74 t
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x1 00000001
0x43 01000011
0xC3 11000011
0x43 01000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x64 d
0x0 00000000
0x3 00000011
0x83 10000011
0x3 00000011
0x64 d
0x61 a
0x1 00000001
0x80 10000000
0x0 00000000
0x83 10000011
0x83 10000011
0x83 10000011
0x43 01000011
0xC0 11000000
0x0 00000000
0x53 S
0x75 u
0x1 00000001
0x1 00000001
0x1 00000001
0x43 01000011
0xC3 11000011
0x75 u
0x45 E
0x45 E
0x45 E
0x45 E
You can see the last address was a Success with character 0x53 indicating an S character. After hours of comparing my sketch output with my C code output I was finally able to get the system to work and the output looked like this:
0x75 u
0xd3 11010011
0x43 01000011
0x03 00000011
0x43 01000011
0xc3 11000011
0x75 u
0x74 t
0xc3 11000011
0xc3 11000011
0x83 10000011
0x03 00000011
0x74 t
0x64 d
0x01 00000001
0x03 00000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x01 00000001
0x43 01000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x00 00000000
0x43 01000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x64 d
0x00 00000000
0x03 00000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x64 d
0x00 00000000
0x03 00000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x64 d
0x00 00000000
0x03 00000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x64 d
0x01 00000001
0x03 00000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x00 00000000
0x43 01000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x44 D
0x61 a
0x03 00000011
0x90 10010000
0xc0 11000000
0xc3 11000011
0xc3 11000011
0xc3 11000011
0xc3 11000011
0x43 01000011
0xd0 11010000
0xd0 11010000
0x61 a
0x46 F
0x75 u
0xf3 11110011
0x43 01000011
0x03 00000011
0x43 01000011
0xc3 11000011
0x75 u
0x75 u
0xd3 11010011
0x43 01000011
0x03 00000011
0x43 01000011
0xc3 11000011
0x75 u
0x74 t
0xc3 11000011
0xc3 11000011
0x83 10000011
0x03 00000011
0x74 t
0x64 d
0x01 00000001
0x03 00000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x01 00000001
0x43 01000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x01 00000001
0x43 01000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x00 00000000
0x43 01000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x64 d
0x01 00000001
0x03 00000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x01 00000001
0x43 01000011
0x43 01000011
0xc3 11000011
0x43 01000011
0x64 d
0x64 d
0x00 00000000
0x43 01000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x64 d
0x00 00000000
0x03 00000011
0x03 00000011
0x83 10000011
0x03 00000011
0x64 d
0x44 D
0x61 a
0x03 00000011
0x90 10010000
0xc0 11000000
0x83 10000011
0x83 10000011
0x83 10000011
0x83 10000011
0x43 01000011
0xd0 11010000
0xd0 11010000
0x61 a
0x53 S
0x75 u
0xf3 11110011
0x63 01100011
0x03 00000011
0x43 01000011
0xc3 11000011
0x75 u
0x45 E
0x45 E
0x45 E
0x45 E
I created a simple python code to parse all my information because I couldn’t understand the entire output.
import pandas as pd
import numpy as np
import sys
fname = sys.argv[1]
infname = sys.argv[2]
analysisflag = sys.argv[3]
if analysisflag=='0':
 f = open(infname,'r')
 readout = f.readline()
 readout = readout.split('\n')[0]
 readout = readout.split(" ")
elif analysisflag=='1':
 readout = pd.read_csv(infname,delimiter='\n',header=None)[0].values.tolist()
else:
 sys.exit();
readout = np.array(['0x'+x for x in readout])
specialchars = np.array(['D','E','S','F','t','u','d','a','p','A'])
specialhex = [hex(ord(x)) for x in specialchars]
binlist = np.array([format(int(x,16),'08b') for x in readout])
for idx,hexOI in enumerate(specialhex):
 locs = np.where(readout==hexOI)[0]
 if len(locs)>0:
 binlist[locs]=specialchars[idx]
df=pd.DataFrame()
df['hexval']=readout
df['binval']=binlist
df.to_csv(fname+'.csv',header=None,index=None,sep='\t')
This allowed me to parse the output from both the sketch and minicom (remember to turn on linewrap when copying the output for minicom), with some manual massaging of the data. This is where having ‘E’ multiple times helped because then I would just look for a hex value that was repeated 4 times in a row and I knew to copy till then. Also, the reason I chose hex was because writing out the character for 0x1 results in some garbage characters. Having the outputs was a big deal because it made it so much easier for me to locate issues such as in the C code where I wasn’t correctly creating the stop bit.
Now I could prove that I could get things to work, I tried to make a loop that went through and tried to find the right address. The only difference in the code for sketch file is that there is a for loop that goes through every value until ack==0 from which it says that it has found the address.
int sda = 7;//9;
int scl = 6;//8;
void setup() {
 // put your setup code here, to run once:
 pinMode(sda,OUTPUT);
 digitalWrite(sda,HIGH);
 pinMode(scl,OUTPUT);
 digitalWrite(scl,HIGH);
 Serial.begin(9600);
 unsigned int len = 0;
 sendStop();
 int counter = 0;
 int ack=1;
 int idx = 0;
 for(counter=0;counter<128;counter++){
  sendStart();
  ack=1;
  Serial.print("Counter: ");Serial.println(counter);
  String binval = String(counter,BIN);
  String eightbitval = create8bit(binval);
  sendStr(eightbitval);
  Serial.print("Data: ");Serial.println(eightbitval);
  Serial.println();
  ack = getAck();
  if(ack == LOW){
  Serial.print("Address: ");
  Serial.println(counter,HEX);
  break;
  }
  sendStop();
  Serial.println();
  Serial.println("Done with one cycle");
 } 
 Serial.println("Exiting Setup!");
}

void loop() {
 // put your main code here, to run repeatedly:
}

String create8bit(String binval){
 int len = binval.length();
 int idx=0;int idx2=0;
 String largebinval = "0000000"+binval;
 int fullLen = largebinval.length();
 String bin8bitval = "00000000";
 for(idx=fullLen-7;idx<fullLen;idx++){
 bin8bitval[idx2] = largebinval[idx];idx2++;
 }
 return bin8bitval;
}

void sendStr(String addrstr){
 int counter = 0;
 Serial.print("Data: ");
 for(counter=0;counter<8;counter++){
 sendData(addrstr[counter]-'0');
 }
 Serial.println();
}

void sendZeros(int diff){
 int i = 0;
 for(i=0;i<diff;i++){
 sendData((unsigned int)('0'-'0'));
 //Serial.println("in for loop");
 }
}

unsigned int getAck(){
 unsigned int ack=1;
 delay(10);
 pinMode(sda,INPUT);
 delay(10);
 digitalWrite(scl,HIGH);
 delay(10);
 ack = digitalRead(sda);
 delay(10);
 digitalWrite(scl,HIGH);
 delay(10);
 digitalWrite(scl,LOW);
 delay(10);
 Serial.print("ack: ");
 Serial.println(ack);
 pinMode(sda,OUTPUT);
 return ack;
}

void sendData(unsigned int data){
 //Serial.print("Data :");
 Serial.print(data,HEX);
 if(data==0){
 digitalWrite(sda,LOW);}
 else{
 digitalWrite(sda,HIGH);}
 delay(10);
 digitalWrite(scl,HIGH);
 delay(10);
 digitalWrite(scl,LOW);
 delay(10);
}

void sendStop(){
 digitalWrite(scl,LOW);
 delay(10);
 digitalWrite(sda,LOW);
 delay(10);
 digitalWrite(sda,HIGH);
 delay(10);
 digitalWrite(scl,HIGH);
 delay(10);
}

void sendStart(){
 digitalWrite(scl,HIGH);
 delay(10);
 digitalWrite(sda,LOW);
 delay(10);
 digitalWrite(scl,LOW);
 delay(10);
}
The only things that were mainly added were:
 for(counter=0;counter<128;counter++){
 sendStart();
 ack=1;
 Serial.print("Counter: ");Serial.println(counter);
 String binval = String(counter,BIN);
 String eightbitval = create8bit(binval);
 sendStr(eightbitval);
 Serial.print("Data: ");Serial.println(eightbitval);
 Serial.println();
 ack = getAck();
 if(ack == LOW){
 Serial.print("Address: ");
 Serial.println(counter,HEX);
 break;
 }
 sendStop();
 Serial.println();
 Serial.println("Done with one cycle");
 } 
 Serial.println("Exiting Setup!");
}

void loop() {
 // put your main code here, to run repeatedly:
}

String create8bit(String binval){
 int len = binval.length();
 int idx=0;int idx2=0;
 String largebinval = "0000000"+binval;
 int fullLen = largebinval.length();
 String bin8bitval = "00000000";
 for(idx=fullLen-7;idx<fullLen;idx++){
 bin8bitval[idx2] = largebinval[idx];idx2++;
 }
 return bin8bitval;
}
All the code did was create a for loop and make sure that I am always sending an 8 bit value. It was the samething for the C code where I had to build up a function that converts a number to a binary string and sends that through.
 for(i=0;i<256;i++){
 USART_transmit(i);
 ack=1;
 sendStart();
 convert_to_binary(i,addrstr);
 //sendStr(addrstr1);
 sendStr(addrstr);
 //USART_transmit('D');
 ack = getAck();
 checkAck(ack);
 sendStop();
 if(ack==0){
 USART_transmit('A');
 USART_transmit('A');
 USART_transmit('A');
 USART_transmit(i);
 break;
 }
 //second address
 DDRD = 0;
 PORTD = 0;
 //DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (0<<SDA)|(1<<SCL)|(1<<PORTD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 //morepininfo();
 sendStop(); 
 }
void convert_to_binary(int num, char binstr[9]){
 char temp[9] = "00000000";
 int rem=0;
 int quotient = 0;
 int idx = 7;
 while(num/2!=0){
 rem = num%2;
 num=num/2;
 //printf("quotient: %d\n",num);
 //printf("rem: %d\n",rem);
 //printf("char: %c\n",(char)rem+'0');
 temp[idx] = (char)(rem+'0');
 idx--;
 }
 
 rem = num%2;
 temp[idx] = (char)(rem+'0');
 //printf("rem: %d\n",(char)rem);
 //printf("char: %c\n",(char)(rem+'0'));
 strncpy(binstr,temp,9);
}
The converter works by dividing the number by 2 and taking the remainder until the quotient equal 0. Wikihow has a great explanation of this. This resulted in slight refactoring of the code.
bit_bang_MS5607.c
#include<stdlib.h>
#include<stdio.h>
#include<avr/interrupt.h>
#include<avr/io.h> 
#include<util/delay.h>
#include<string.h>

#define FOSC 16000000 
#define BAUD 9600
#define MYUBRR FOSC/16/BAUD-1 
//#define MYUBRR 103
#define SCL PORTD7
#define SDA PORTD6

void USART_init(unsigned int ubrr);
void USART_transmit(char data);
unsigned char USART_recieve(void);
unsigned int getAck();
void sendStart();
void sendStop();
void checkAck(unsigned int ack);
void sendData(unsigned char data);
void sendStr(char data[8]);
void morepininfo();
void convert_to_binary(int num, char binstr[9]);

int main(){

char addrstr[9] = "00000000";// = "11101100";//236
 char* addrstr1 = "11000010";//194
 char* addrstr2 = "10101010";
 unsigned int ack;
 USART_init(MYUBRR);
 _delay_ms(10000);
 DDRD = 0;
 PORTD = 0;
 //DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (0<<SDA)|(1<<SCL)|(1<<PORTD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 //first address
 sendStop();
 int i = 0;
 for(i=0;i<256;i++){
 USART_transmit(i);
 ack=1;
 sendStart();
 convert_to_binary(i,addrstr);
 //sendStr(addrstr1);
 sendStr(addrstr);
 //USART_transmit('D');
 ack = getAck();
 checkAck(ack);
 sendStop();
 if(ack==0){
 USART_transmit('A');
 USART_transmit('A');
 USART_transmit('A');
 USART_transmit(i);
 break;
 }
 //second address
 DDRD = 0;
 PORTD = 0;
 //DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (0<<SDA)|(1<<SCL)|(1<<PORTD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 //morepininfo();
 sendStop(); 
 }
/* sendStart();
 convert_to_binary(194,addrstr);
 sendStr(addrstr1);
 //USART_transmit('D');
 ack = getAck();
 checkAck(ack);
 sendStop();*/
 //second address
 DDRD = 0;
 PORTD = 0;
 //DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (0<<SDA)|(1<<SCL)|(1<<PORTD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 //morepininfo();
 sendStop();
 sendStart();
 convert_to_binary(236,addrstr);
 //sendStr(addrstr);
 sendStr(addrstr);
 //USART_transmit('D');
 ack=getAck();
 checkAck(ack);
 sendStop();
 //USART_transmit('E'); 
 //USART_transmit('E');
 //USART_transmit('E');
 //USART_transmit('E');

while(1){
 _delay_ms(10);
 PORTD |= (1<<PORTD4);
 _delay_ms(10);
 PORTD ^= (1<<PORTD4);
 _delay_ms(10);
 //USART_transmit(PIND+65);
 _delay_ms(5000);
 if ((PIND)&(1<<SDA)){
 //USART_transmit('i');
 _delay_ms(10);
 PORTD ^= (1<<SCL);
 }
 ////USART_transmit(97);
 //data = USART_recieve();
 ////USART_transmit(data);
 //_delay_ms(1000);
}
}

void convert_to_binary(int num, char binstr[9]){
 char temp[9] = "00000000";
 int rem=0;
 int quotient = 0;
 int idx = 7;
 while(num/2!=0){
 rem = num%2;
 num=num/2;
 //printf("quotient: %d\n",num);
 //printf("rem: %d\n",rem);
 //printf("char: %c\n",(char)rem+'0');
 temp[idx] = (char)(rem+'0');
 idx--;
 }

 rem = num%2;
 temp[idx] = (char)(rem+'0');
 //printf("rem: %d\n",(char)rem);
 //printf("char: %c\n",(char)(rem+'0'));
 strncpy(binstr,temp,9);
}




void morepininfo(){
 //USART_transmit('p');
 //USART_transmit(PORTD);
 //USART_transmit(DDRD);
 //USART_transmit('p');
}




void checkAck(unsigned int ack){
 if(ack == 0){
 //USART_transmit('S');
 }
 else{
 //USART_transmit('F');
 }
}

void sendStr(char addr[8]){
 int i = 0;
 for(i=0;i<8;i++){
 sendData(addr[i]);
 }
}

void sendStart(){
 //USART_transmit('t');
 //USART_transmit(PIND);
 PORTD |= (1<<SCL); //setting scl low
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (0<<SDA)|(1<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (0<<SDA)|(0<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 //USART_transmit('t');
}

void sendStop(){
 //USART_transmit('u');
 //USART_transmit(PIND);
 PORTD = (0<<SCL)|(1<<SDA);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (0<<SDA)|(0<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (1<<SDA)|(0<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (1<<SDA)|(1<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 //USART_transmit('u');
}




void sendData(unsigned char data){
 //USART_transmit('d');
 int val = data-'0';
 //USART_transmit(val);
 //USART_transmit(PIND);
 PORTD = (val<<SDA)|(0<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (val<<SDA)|(1<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (val<<SDA)|(0<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 //USART_transmit('d');
}

unsigned int getAck(){
 //USART_transmit('a');
 unsigned int ack=1;
 _delay_ms(10);
 //USART_transmit(PIND);
 //convert PIND6 to input
 DDRD = (0<<DDD6)|(1<<DDD7)|(1<<DDD4);
 //PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4); 
 PORTD = (1<<SDA)|(1<<SCL);
 //USART_transmit(DDRD);
 //USART_transmit(PORTD);
 _delay_ms(10);
 PORTD |= (1<<SCL);
 //_delay_ms(10000);
 //USART_transmit(PIND);
 _delay_ms(10);
 //USART_transmit(PIND);
 if(~(PIND)&(1<<SDA)){
 ack = 0;}
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD |= (1<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 PORTD = (0<<SCL);
 _delay_ms(10);
 //USART_transmit(PIND);
 //set pin6 to output
 DDRD = (1<<DDD6)|(1<<DDD7)|(1<<DDD4);
 PORTD = (1<<SDA)|(1<<SCL)|(1<<PORTD4);
 _delay_ms(10);
 //USART_transmit(DDRD);
 //USART_transmit(PORTD);
 //USART_transmit('a');
 return ack;
}




void USART_init(unsigned int ubrr){
 UCSR0B = (1<<TXEN0)|(1<<RXEN0);

//choose asynchronous mode
 UCSR0C = 0;
 UCSR0C = (0<<UMSEL01)|(0<<UMSEL00);
 UCSR0C |= (1<<UCSZ01)|(1<<UCSZ00);
 UCSR0C |= (1<<USBS0);
 /*//setting UBRR TO 103 for 9600 baudrate
 UBRR0H = 0;
 UBRR0L = 0b01100111; */
 UBRR0H = (unsigned char)(ubrr>>8);
 UBRR0L = (unsigned char)ubrr;
}

void USART_transmit(char data){
 char temp='a';//initialize to a certain character
 //while(temp){
 //temp = *(data++);//go through each character one by one until all the data has been put out
 while( !(UCSR0A & (1<<UDRE0)) ) //
 ;
 //UDR0 = temp;
 UDR0 = data;
 //}
 _delay_ms(10);
}

unsigned char USART_recieve(void){
 /*wait for data to be recieved, this is important because the data register has to get filled up and a flag gets created when a stop bit is set high*/
 while(!(UCSR0A & (1<<RXC0) ))
 ;
 //once the stop bit is set high then read directly from UDRN
 return UDR0;
}

Bot the C code and sketch worked and were able to stop at the appropriate address, which was great. Great exercise, but took a while to get things started. I would really recommend the approach I took, which was to start with sketch and then go to C code because it atleast makes sure if your logic is right. Here is the code to convert from integer to binary as a separate C program, because I like breaking things down and then combining them.

binconv.c

#include<stdio.h>
#include<stdlib.h>
#include<string.h>

void convert_to_binary(int num,char binstr[9]);

int main(){
 int i = 0;
 char binstr[9]= "00000000";
 int pt1=20;int pt2=33;
 //for(i=0;i<128;i++){
 printf("\n20\n");
 convert_to_binary(pt1,binstr);
 printf("Bin Str: %s\n",binstr);
 printf("\n33\n");
 char binstr1[9]= "00000000"; 
 convert_to_binary(pt2,binstr);
 printf("Bin Str: %s\n",binstr);
 printf("\n");
 //}
}

void convert_to_binary(int num, char binstr[9]){
 char temp[9] = "00000000";
 int rem=0;
 int quotient = 0;
 int idx = 7;
 while(num/2!=0){
 rem = num%2;
 num=num/2;
 printf("quotient: %d\n",num);
 printf("rem: %d\n",rem);
 printf("char: %c\n",(char)rem+'0');
 temp[idx] = (char)(rem+'0');
 idx--;
 }
 
 rem = num%2;
 temp[idx] = (char)(rem+'0');
 printf("rem: %d\n",(char)rem);
 printf("char: %c\n",(char)(rem+'0'));
 strncpy(binstr,temp,9);
}
That’s all folks! Let me know if you have any questions! My code can be found on my github. Here are a list of my useful links:

UART on arduino uno in C

 

This has been an information dump, I am trying to write out everything that I have been doing for a past couple of weeks. This post  is on USART which stands for Universal Synchronous/Asynchronous Receiver and Transmitter (I will be focusing on UART), which is used to communicate arduino uno with my computer. There is a great explanation of UART works from circuit basics. Why do I want to communicate with my computer? As your programs get sufficiently complex, it becomes harder and harder to debug the code because there are so many things that can go wrong and it is nice to have print statements that tell you the state of a variable (I do this a lot). In order to get up and running I had to learn how to use, USART because you might not always have a debugger handy. I will walk through the code to help everyone understand what I did.

#include<stdlib.h>
#include<stdio.h>
#include<avr/interrupt.h>
#include<avr/io.h> 
#include<util/delay.h>

#define FOSC 16000000 
#define BAUD 9600
#define MYUBRR FOSC/16/BAUD-1 
//#define MYUBRR 103

void USART_init(unsigned int ubrr);
void USART_transmit(char data);
unsigned char USART_recieve(void);

int main(){
 char data;//"Hello!";
 USART_init(MYUBRR);
 while(1){
 //USART_transmit(97);
 data = USART_recieve();
 USART_transmit(data);
 //_delay_ms(1000);
}
}

void USART_init(unsigned int ubrr){
 UCSR0B = (1<<TXEN0)|(1<<RXEN0);

//choose asynchronous mode
 UCSR0C = 0;
 UCSR0C = (0<<UMSEL01)|(0<<UMSEL00);
 UCSR0C |= (1<<UCSZ01)|(1<<UCSZ00);
 UCSR0C |= (1<<USBS0);
 /*//setting UBRR TO 103 for 9600 baudrate
 UBRR0H = 0;
 UBRR0L = 0b01100111; */
 UBRR0H = (unsigned char)(ubrr>>8);
 UBRR0L = (unsigned char)ubrr;
}

void USART_transmit(char data){
 char temp='a';//initialize to a certain character
 //while(temp){
 //temp = *(data++);//go through each character one by one until all the data has been put out
 while( !(UCSR0A & (1<<UDRE0)) ) //
 ;
 //UDR0 = temp;
 UDR0 = data;
 //}
}

unsigned char USART_recieve(void){
 /*wait for data to be recieved, this is important because the data register has to get filled up and a flag gets created when a stop bit is set high*/
 while(!(UCSR0A & (1<<RXC0) ))
 ;
 //once the stop bit is set high then read directly from UDRN
 return UDR0;
}

The program is simple, you have first initialize the USART and then I am sending data to USART which then gets sent back to make sure the code works.

The first part is initializing the USART which is explained by the USART_init function which is in section 20.5 of the ATMEGA328P manual. The code is a copy of:

Screenshot from 2017-09-03 13:06:41

void USART_init(unsigned int ubrr){
 UCSR0B = (1<<TXEN0)|(1<<RXEN0);

//choose asynchronous mode
 UCSR0C = 0;
 UCSR0C = (0<<UMSEL01)|(0<<UMSEL00);
 UCSR0C |= (1<<UCSZ01)|(1<<UCSZ00);
 UCSR0C |= (1<<USBS0);
 /*//setting UBRR TO 103 for 9600 baudrate
 UBRR0H = 0;
 UBRR0L = 0b01100111; */
 UBRR0H = (unsigned char)(ubrr>>8);
 UBRR0L = (unsigned char)ubrr;
}

One of the first things you are supposed to do is set the baud rate, but what I did instead is setup the pins to transmit and receive. The next step I did was choose if are going to run in synchronous mode or asynchronous mode, and for the purpose of this post I chose asynchronous mode. I chose asynchronous mode because I was just trying get things up and running. There is a great post on EDN that explains the difference between UART and USART (it essentially boils down to the fact that USART has everything synchronized to a clock so it can run at higher baud rates). I chose it to be asynchronous by setting UMSEL01 and UMSEL00 to 0. Once that was completed I defined the size of the stop bit (done by setting the line to high) and I did it for 2 bits (don’t really fully comprehend why this matters, but it seems to be related to running at higher baud rates) by setting UCSZ01 AND UCSZ00 to both 1. The next step was to have no parity bit, basically it means there is no error checking to make sure the data was sent properly. This is outlined in p.194 of the ATMEGA32P manual. Once that was completed I chose a baud rate to run at and placed that value into 2 8bit registers. Once this was completed I moved onto receive and transmitting data:

void USART_transmit(char data){
 char temp='a';//initialize to a certain character
 //while(temp){
 //temp = *(data++);//go through each character one by one until all the data has been put out
 while( !(UCSR0A & (1<<UDRE0)) ) //
 ;
 //UDR0 = temp;
 UDR0 = data;
 //}
}

unsigned char USART_recieve(void){
 /*wait for data to be recieved, this is important because the data register has to get filled up and a flag gets created when a stop bit is set high*/
 while(!(UCSR0A & (1<<RXC0) ))
 ;
 //once the stop bit is set high then read directly from UDRN
 return UDR0;
}

Yes, I noticed that I misspelled receive. The code is simple for transmit, the first that happens before sending the data we wait to see if in the register UCSR0A or USART Control and Status Register UDRE0 is set to 1. If it set to one that means the buffer is ready to be written to.

screenshot-from-2017-09-03-140700.png

UDR0 is only written to once UDRE0 flag is set to 1 otherwise the buffer still have data in it. The receive function does exactly the same, but instead of writing to UDR0 it read from UDR0 as outline by

screenshot-from-2017-09-03-141137.png

It reads in the data from UDR0 only when RXC0 flag is set. You can see it working by doing the following, first set up minicom with the -s flag and then go to “Serial por setup” and then choose the device that is your uno, and the choose the communication parameters which is under “Bps/Par/Bits” and choose current to be 9600N2 where 9600 is the baud rate N is no parity bit and 2 means 2 stop bits and then I hit “Exit” NOT “Exit from Minicom”. In another terminal I would type

user@user-Bonobo-Extreme:~/git-repos/arduino/arduino_uno/manual_I2C$ ls /dev/ttyACM0 
/dev/ttyACM0
user@user-Bonobo-Extreme:~/git-repos/arduino/arduino_uno/manual_I2C$ echo "a" > /dev/ttyACM0

and when you look at the minicom terminal you should see an “a” character pop up

Welcome to minicom 2.7

OPTIONS: I18n 
Compiled on Jan 1 2014, 17:13:19. 
Port /dev/ttyACM0, 14:20:08 
 
Press CTRL-A Z for help on special keys 
 
a

which it does! So there you go. The code can be found on my github.

As always here is a list of my useful links:

http://www.edn.com/electronics-blogs/embedded-basics/4440395/USART-vs-UART–Know-the-difference

http://www.unm.edu/~zbaker/ece238/slides/UART.pdf

http://www.circuitbasics.com/basics-uart-communication/

https://learn.sparkfun.com/tutorials/serial-communication/rules-of-serial

Using ISR on arduino uno in C

This will be about how to use ISR(interrupt service request) in arduino uno. The code will trip a ISR which means once the ISR is tripped, the code stops and executes the ISR and then moves on. I used header files again for the timers and then wrote code that works with the ISR.
The header file is defined as follows:

#include<stdio.h>
#include<stdlib.h>

//Register variables

#define OCR1AL_reg (*(volatile unsigned char *)(0x88)) //output compare low 8 bits for Counter 1
#define OCR1AH_reg (*(volatile unsigned char *)(0x89)) //output compare high 8 bits for Counter 1

#define TCCR1B_reg (*(volatile unsigned char *)(0x81)) //Timer/Counter0 Control Register B 
#define TCCR1A_reg (*(volatile unsigned char *)(0x80)) //Timer/Counter0 Control Register A

#define TIMSK1_reg (*(volatile unsigned char *)(0x6F)) //Timer/Counter0 Interrupt Mask Register

#define TIFR1_reg (*(volatile unsigned char *)(0x36)) //Timer/Counter0 Interrupt Flag Register

#define TCNT1H_reg (*(volatile unsigned char *)(0x85)) //Timer/Counter0 Interrupt Flag Register
#define TCNT1L_reg (*(volatile unsigned char *)(0x84)) //Timer/Counter0 Interrupt Flag Register

//setting up PBn
#define DDRB_reg (*(volatile unsigned char *)(0x24)) //Sets up the DDR for B
#define PORTB_reg (*(volatile unsigned char *)(0x25)) //sets up port B
#define PINB_reg (*(volatile unsigned char *)(0x23)) //to be able to read PINB
#define PCICR_reg (*(volatile unsigned char *)(0x68))

//setting up the global interrupt register
#define SREG_reg (*(volatile unsigned char *)(0x5F)) //Status register used for disabling and enabling global interrupts

#define FCLK_atmega32u4 8000000 //clock speed

//setting up TCCR1A control register
int COM1A1_val = 7;//0b10000000;
int COM1A0_val = 6;//0b01000000;
int COM1B1_val = 5;//0b00100000;
int COM1B0_val = 4;//0b00010000;
int COM1C1_val = 3;//0b00001000;
int COM1C0_val = 2;//0b00000100;
int WGM11_val = 1;//0b00000010;
int WGM10_val = 0;//0b00000001;

//setting up TCCR1B control register
int ICNC1_val = 7;//0b10000000;
int ICES1_val = 6;//0b01000000;
int WGM13_val = 4;//0b00010000;
int WGM12_val = 3;//0b00001000;
int CS12_val = 2;//0b00000100;
int CS11_val = 1;//0b00000010;
int CS10_val = 0;//0b00000001;


int Bit_0 = 0;
int Bit_1 = 1;
int Bit_2 = 2;
int Bit_3 = 3;
int Bit_4 = 4;
int Bit_5 = 5;
int Bit_6 = 6;
int Bit_7 = 7;

//setting up TIMSK1 register
int OCIE1A_val = 1;
int ICIE1_val = 5;

//unsigned char prescale_8 = CS11_val;
//unsigned char prescale_64 = CS10_val|CS11_val;
//unsigned char prescale_256 = CS12_val;
//unsigned char prescale_1024= CS12_val|CS10_val;

The time the code is done slightly differently where I have assigned int values to the bits instead of binary values, this allows me to do (1<<Bit_0). You can see that it is pretty similar to the header file in my previous post. I will start talking about to use the ISR and how the timers are being set up.

#include "arduino_pin_header.h"
#include<stdio.h>
#include<stdlib.h>
#include<avr/interrupt.h>

int x = 0;

int main(){

DDRB_reg = 0;
 PORTB_reg = 0;
 DDRB_reg = (1<<Bit_5);
 PORTB_reg = (0<<Bit_5);

//resetting the Timer/Counter1
 TCNT1H_reg = 0;
 TCNT1L_reg = 0; 
 
 //disabling all global interrupts
 SREG_reg = 0;

//defining prescalar

//TCCR1B: ICNC1 ICES1 – WGM13 WGM12 CS12 CS11 CS10
 TCCR1B_reg = 0;
 //TCCR1B_reg = CS11_val; WORKING SETTINGS
 TCCR1B_reg = (1<<CS10_val)|(0<<CS12_val);//0b00000101;//(1<<CS10_val)|(1<<CS12_val);

//setting up PWM mode

//TCCR1A: COM1A1 COM1A0 COM1B1 COM1B0 COM1C1 COM1C0 WGM11 WGM10
 TCCR1A_reg = 0; //this is for waveform generation and CTC setting up mode;
 TCCR1A_reg = (1<<COM1A1);//0b10000000;//(1<<COM1A1_val);
 //TCCR1A_reg = COM1A1_val|WGM10_val; //this is for waveform generation and CTC setting up mode; WORKING SETTINGS

//defining the output compare value which is calculated to be 3906 assuming 8MHz clock speed: OCRNA = (fClk_I/O)/(2*N*fOCRNA) where N is the prescalar fOCRNA is the compare value and fClk_I/O is the clock speed
 //the value calculated for fOCRNA to be 1Hz is ~3096
 //100Hz = 10011 10001000
 //10kHz = 110010
 //100kHz = 
 // (*(volatile unsigned char *)(OCR1AH_reg)) = Bit_3|Bit_2|Bit_1|Bit_0;//0b00001111; //writing to high first which will write to a temp register first
 //OCR1AH_reg = 0;//0b0001100;//0b00001111; //writing to high first which will write to a temp register first
 // (*(volatile unsigned char *)(OCR1AL_reg)) = Bit_6|Bit_1;//0b01000010; //writing to low will write both high and low in the same clock cycle
 //OCR1AL_reg = 0b11001110;//0b01000010; //writing to low will write both high and low in the same clock cycle DEFAULT WORKING SETTINGS
 //OCR1AL_reg = 0b00000101;

OCR1AH_reg = 0b10011100;//0b00000000;
 OCR1AL_reg = 0b01000000;//0b01000000;
 
 TIMSK1_reg = (1<<OCIE1A_val)|(1<<ICIE1_val);//0b00100010;//(1<<ICIE1_val)|(1<<OCIE1A_val);//0b00100010; //writing so that output compare A is set up

//enable global interrupts
 SREG_reg = (1<<Bit_7);
 while(1){}
 return 0;

}

ISR(TIMER1_COMPA_vect){
 
 PORTB_reg = PORTB_reg^(1<<Bit_5);
}

Now you can see a new thing pop up which is ISR(TIMER1_COMPA_vect). I am using Bit 5 because then I can just watch the LED get toggled.

DDRB_reg = 0;
 PORTB_reg = 0;
 DDRB_reg = (1<<Bit_5);
 PORTB_reg = (0<<Bit_5);

//resetting the Timer/Counter1
 TCNT1H_reg = 0;
 TCNT1L_reg = 0; 
 
 //disabling all global interrupts
 SREG_reg = 0;

The first part of the code sets DDRB as output and sets PORTB to 1 which will then set up the LED for the pin or can be used to output to that pin. At the same time, timer1 is set to 0 and all global interrupts are disabled so the code doesn’t stall if the uC suddenly gets an interrupt.

//TCCR1B: ICNC1 ICES1 – WGM13 WGM12 CS12 CS11 CS10
 TCCR1B_reg = 0;
 //TCCR1B_reg = CS11_val; WORKING SETTINGS
 TCCR1B_reg = (1<<CS10_val)|(0<<CS12_val);//0b00000101;//(1<<CS10_val)|(1<<CS12_val);

//setting up PWM mode

//TCCR1A: COM1A1 COM1A0 COM1B1 COM1B0 COM1C1 COM1C0 WGM11 WGM10
 TCCR1A_reg = 0; //this is for waveform generation and CTC setting up mode;
 TCCR1A_reg = (1<<COM1A1);//0b10000000;//(1<<COM1A1_val);

The next step is the prescalar which I described in my previous post and the table for it is show below (found on p.134):

Screenshot from 2017-09-03 02:25:49

This is being set in Counter Register B. The next step sets ups TCCR1A. The first part is setting the TCCR1A to normal setting as shown in the table below:

Screenshot from 2017-09-03 00:42:16

You can see when the mode is set to the the Timer/Counter Mode operation is set to 0 so it is not doing any Phase correction and things like that. What I do set is COM1A1, and this sets up the pins so that in normal mode the normal behavior of PinB5 is taken over by the timer.

Screenshot from 2017-09-03 02:31:33

When COM1A1 is set the the 0C1A is cleared when the timer counts a certain level and the output is set to low. So all this is doing is making OC1A go low.

 OCR1AH_reg = 0b10011100;//0b00000000;
 OCR1AL_reg = 0b01000000;//0b01000000;

This just sets the limit at which the timer will count to until clearing OC1A in this case (or setting OC1A low or digitally think of it being set to 0).

TIMSK1_reg = (1<<OCIE1A_val)|(1<<ICIE1_val);//0b00100010;//(1<<ICIE1_val)|(1<<OCIE1A_val);//0b00100010; //writing so that output compare A is set up

//enable global interrupts
 SREG_reg = (1<<Bit_7);
 while(1){}
 return 0;

The next major line is the TIMSK1_reg which sets up the Timer interrupt mask register and sets up the interrupts that we are interested in with the global interrupts. This sets up interrupts to be enabled on OCIE1A (links up with OC1A pin). It also sets up ICIE1 which sets up input interrupt (meaning an input signal will trip the start of an interrupt. Since we are setting up these interrupts we don’t want to trip an interrupt while you are setting things up to avoid race conditions. So we enable global interrupts to get everything going.

screenshot-from-2017-09-03-024751.png

screenshot-from-2017-09-03-024805.png

The while loop near the end of the main() is very important. What would happen is in the PWM case I don’t need to put the while loop because the timer will keep working independently and gets set once. In the case of the interrupts, the timer has to keep running and every time the interrupt gets called it has to look back on the code but if the code has exited that function has gone away, so it was important to have the while loop at the end or the ISR might only get tripped at all or never at all.

ISR(TIMER1_COMPA_vect){

PORTB_reg = PORTB_reg^(1<<Bit_5);
}

The ISR code is only gets tripped when the OCIE1A is set and it sends it to the ISR function (I would think of it more of a macro). When the ISR we can define which interrupt will cause what behavior and in this case an ISR generated by TIMER1_COMPA_vect will cause PinB5 to get toggled high or low based on its previous state. Finding the right variable that goes with the OC1A took a while but can be found here. The annoying thing about finding those interrupt vectors, is that they are not very obvious. But in short timer gets started –> hits the threshold –> creates an interrupt flag –> sends it to the ISR –> ISR check what caused the request –> sends it directly to the predefined function. I have been told not to spend too much time in ISR because it can create a backlog very quickly.

Here are the links I found useful:

http://www.avrbeginners.net/architecture/timers/timers.html

http://ee-classes.usc.edu/ee459/library/documents/avr_intr_vectors/

My code can be found in my git repo and the name of the c file is ISR_example.c

 

PWM with TIMER1 on arduino uno

Tried to create a PWM with TIMER1 on arduino uno. This was done merely to get an understanding of how to use timers in arduino uno, which should be similar to the SparkFun ProMicro. I will try to go through all the steps as best as I can. I started doing this directly in C, which if you are a beginner like me and don’t want to get specialized to tools to debug issues, I would not recommend using. I would recommend going from sketch and then to C to make sure that your logic makes sense. Let’s start with the header file:

#include<stdio.h>
#include<stdlib.h>




//Register variables

#define OCR1AL_reg (*(volatile unsigned char *)(0x88)) //output compare low 8 bits for Counter 1
#define OCR1AH_reg (*(volatile unsigned char *)(0x89)) //output compare high 8 bits for Counter 1

#define TCCR1B_reg (*(volatile unsigned char *)(0x81)) //Timer/Counter0 Control Register B 
#define TCCR1A_reg (*(volatile unsigned char *)(0x80)) //Timer/Counter0 Control Register A

#define TIMSK1_reg (*(volatile unsigned char *)(0x6F)) //Timer/Counter0 Interrupt Mask Register

#define TIFR1_reg (*(volatile unsigned char *)(0x36)) //Timer/Counter0 Interrupt Flag Register

#define TCNT1H_reg (*(volatile unsigned char *)(0x85)) //Timer/Counter0 Interrupt Flag Register
#define TCNT1L_reg (*(volatile unsigned char *)(0x84)) //Timer/Counter0 Interrupt Flag Register

//setting up PBn
#define DDRB_reg (*(volatile unsigned char *)(0x24)) //Sets up the DDR for B
#define PORTB_reg (*(volatile unsigned char *)(0x25)) //sets up port B
#define PINB_reg (*(volatile unsigned char *)(0x23)) //to be able to read PINB
#define PCICR_reg (*(volatile unsigned char *)(0x68))

//setting up the global interrupt register
#define SREG_reg (*(volatile unsigned char *)(0x5F)) //Status register used for disabling and enabling global interrupts

#define FCLK_atmega32u4 8000000 //clock speed




//setting up TCCR1A control register
unsigned char COM1A1_val = 0b10000000;
unsigned char COM1A0_val = 0b01000000;
unsigned char COM1B1_val = 0b00100000;
unsigned char COM1B0_val = 0b00010000;
unsigned char COM1C1_val = 0b00001000;
unsigned char COM1C0_val = 0b00000100;
unsigned char WGM11_val = 0b00000010;
unsigned char WGM10_val = 0b00000001;

//setting up TCCR1B control register
unsigned char ICNC1_val = 0b10000000;
unsigned char ICES1_val = 0b01000000;
unsigned char WGM13_val = 0b00010000;
unsigned char WGM12_val = 0b00001000;
unsigned char CS12_val = 0b00000100;
unsigned char CS11_val = 0b00000010;
unsigned char CS10_val = 0b00000001;




unsigned char Bit_0 = 0b00000001;
unsigned char Bit_1 = 0b00000010;
unsigned char Bit_2 = 0b00000100;
unsigned char Bit_3 = 0b00001000;
unsigned char Bit_4 = 0b00010000;
unsigned char Bit_5 = 0b00100000;
unsigned char Bit_6 = 0b01000000;
unsigned char Bit_7 = 0b10000000;

//unsigned char prescale_8 = CS11_val;
//unsigned char prescale_64 = CS10_val|CS11_val;
//unsigned char prescale_256 = CS12_val;
//unsigned char prescale_1024= CS12_val|CS10_val;

I made this header file so that I don’t have to use any of the arduino libraries. This was more of a learning experience for me to understand how the registers are used and how the values are set on those register. You can see the register are defined as “volatile” and are after #define to make it easier for me to understand, look at my earlier blog post to understand what the volatile and then dereferencing it. The next set of unsigned char basically assign different values on the registers. These value setup the register to run in different modes which is especially important for setting up pins to work with timers. The next set of code shows how the timers is being created and PWMs are being set-up.

#include "arduino_pin_header.h"
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<math.h>

void setOCRnxRegisters(unsigned int OCnumber);
void setPWMFrequency(double PWMFrequency);

int main(){

 DDRB_reg = 0;
 PORTB_reg = 0;
 DDRB_reg = Bit_1;
 PORTB_reg = Bit_1;

//resetting the Timer/Counter1
 TCNT1H_reg = 0;
 TCNT1L_reg = 0; 
 
 //disabling all global interrupts
 SREG_reg = 0;

//defining prescalar

//TCCR1B: ICNC1 ICES1 – WGM13 WGM12 CS12 CS11 CS10
 TCCR1B_reg = 0;
 //TCCR1B_reg = CS11_val; WORKING SETTINGS
 TCCR1B_reg = WGM13_val;

//setting up PWM mode

//TCCR1A: COM1A1 COM1A0 COM1B1 COM1B0 COM1C1 COM1C0 WGM11 WGM10
 TCCR1A_reg = 0; //this is for waveform generation and CTC setting up mode;
 TCCR1A_reg = WGM10_val|COM1A0_val;
 //TCCR1A_reg = COM1A1_val|WGM10_val; //this is for waveform generation and CTC setting up mode; WORKING SETTINGS

//defining the output compare value which is calculated to be 3906 assuming 8MHz clock speed: OCRNA = (fClk_I/O)/(2*N*fOCRNA) where N is the prescalar fOCRNA is the compare value and fClk_I/O is the clock speed
 //the value calculated for fOCRNA to be 1Hz is ~3096
 //100Hz = 10011 10001000
 //10kHz = 110010
 //100kHz = 
 // (*(volatile unsigned char *)(OCR1AH_reg)) = Bit_3|Bit_2|Bit_1|Bit_0;//0b00001111; //writing to high first which will write to a temp register first
 //OCR1AH_reg = 0;//0b0001100;//0b00001111; //writing to high first which will write to a temp register first
 // (*(volatile unsigned char *)(OCR1AL_reg)) = Bit_6|Bit_1;//0b01000010; //writing to low will write both high and low in the same clock cycle
 //OCR1AL_reg = 0b11001110;//0b01000010; //writing to low will write both high and low in the same clock cycle DEFAULT WORKING SETTINGS
 //OCR1AL_reg = 0b00000101;

setPWMFrequency(100000);

TIMSK1_reg = 0;//0b00100010; //writing so that output compare A is set up

//enable global interrupts
 SREG_reg = Bit_7;

return 0;

}


void setPWMFrequency(double PWMFrequency){
 //focn = fclk/(2*N*TOP) where N is the prescalar
 //TOP = fclk/(2*N*focn)
 printf("Frequency of interest is %.2f\n",PWMFrequency);
 //defining a bunch of prescalars
 int prescalar[5] = {1,8,64,256,1024};
 unsigned char prescalers[5]= {CS10_val,CS11_val,CS10_val|CS11_val,CS12_val,CS12_val|CS10_val};

int x = 0;
 unsigned int OCnumber=0;
 for(x=0;x<5;x++){
 OCnumber = FCLK_atmega32u4/(2*prescalar[x]*PWMFrequency);
 if(OCnumber <= pow(2,16)-1){
 printf("The OC number is %d\n",OCnumber);
 TCCR1B_reg |= prescalers[x];
 printf("Prescalar chose is %d\n",prescalers[x]);
 
 break;
 }
 }
 setOCRnxRegisters(OCnumber); 
}

void setOCRnxRegisters(unsigned int OCnumber){
 //convert a 16bit number into 2 8 bit numbers
 uint8_t binarylow = OCnumber & 0xff;
 uint8_t binaryhigh = (OCnumber >> 8);
 OCR1AH_reg = binaryhigh;
 OCR1AL_reg = binarylow;
}

The first thing we want to note according to p.82 of the ATMEGA328p manual PB1 is linked to the OC1A (Timer1/Counter1 Output Compare Match A Output).

 DDRB_reg = 0; 
 PORTB_reg = 0; 
 DDRB_reg = Bit_1; 
 PORTB_reg = Bit_1;

When looking at p.83 of the  it shows that in order to use OC1A we have to set PB1 to output hence required DDRB1 and PORTB1 be set to 1.

//resetting the Timer/Counter1
 TCNT1H_reg = 0;
 TCNT1L_reg = 0; 
 
 //disabling all global interrupts
 SREG_reg = 0;

//defining prescalar

//TCCR1B: ICNC1 ICES1 – WGM13 WGM12 CS12 CS11 CS10
 TCCR1B_reg = 0;
 //TCCR1B_reg = CS11_val; WORKING SETTINGS
 TCCR1B_reg = WGM13_val;


//setting up PWM mode

//TCCR1A: COM1A1 COM1A0 COM1B1 COM1B0 COM1C1 COM1C0 WGM11 WGM10
 TCCR1A_reg = 0; //this is for waveform generation and CTC setting up mode;
 TCCR1A_reg = WGM10_val|COM1A0_val;

Timer1 is constantly counting and we don’t want it to hit the threshold we are interested in so in order to avoid that we start timer1 at 0 and at the same time remove all interrupts. We disable interrupts, because if we didn’t do that, it would cause the uC to finish the interrupt before moving on to the next step. The next step was setting up the Timer/Counter Register 1A and 1B, to define the timer behavior. The first step is to define how the waveform will work which is set by WGM13 and WGM10 which is in page 132 of the ATMEGA32P manual. The sets the PWM with Phase and Frequency correct(I am not sure of all the differences yet, but it seems to work) or mode 9. The TOP is set by OCR1A which means when the counter hits a certain threshold then it will output to that OCR1A pin.

screenshot-from-2017-09-03-004216.png

Screenshot from 2017-09-03 00:45:57

Table 16-3 defines that when the counter hits the a compare level then it will toggle OC1A when the WGM13:0 = 9 or (mode 9) which is what we are using. So the waveform in mode 9 will cause OC1A to toggle.

 setPWMFrequency(100000);

This goes to the following function:

void setPWMFrequency(double PWMFrequency){
 //focn = fclk/(2*N*TOP) where N is the prescalar
 //TOP = fclk/(2*N*focn)
 printf("Frequency of interest is %.2f\n",PWMFrequency);
 //defining a bunch of prescalars
 int prescalar[5] = {1,8,64,256,1024};
 unsigned char prescalers[5]= {CS10_val,CS11_val,CS10_val|CS11_val,CS12_val,CS12_val|CS10_val};

int x = 0;
 unsigned int OCnumber=0;
 for(x=0;x<5;x++){
 OCnumber = FCLK_atmega32u4/(2*prescalar[x]*PWMFrequency);
 if(OCnumber <= pow(2,16)-1){
 printf("The OC number is %d\n",OCnumber);
 TCCR1B_reg |= prescalers[x];
 printf("Prescalar chose is %d\n",prescalers[x]);
 
 break;
 }
 }
 setOCRnxRegisters(OCnumber); 
}

void setOCRnxRegisters(unsigned int OCnumber){
 //convert a 16bit number into 2 8 bit numbers
 uint8_t binarylow = OCnumber & 0xff;
 uint8_t binaryhigh = (OCnumber >> 8);
 OCR1AH_reg = binaryhigh;
 OCR1AL_reg = binarylow;
}

The first step is to create a prescalar which can be explained in a very simple way. Let’s say a clock moves at a certain frequency, it moves 1 tick per second. The fastest the clock the can move is 1 tick per second. Now let’s say you want to count 10s as 1 time unit for another application. The prescalar essentially says that 10 ticks at 1 ticks per second will count for 1 time unit. There are multiple different prescalars based on how low of a frequency you want to count at. These are done by prescalars array. The for loops goes through every prescalar and tries to figure out which prescalar can be used.

OCnumber = FCLK_atmega32u4/(2*prescalar[x]*PWMFrequency);

This is based on p.128 of the ATMEGA328p manual:

Screenshot from 2017-09-03 00:58:55

All I did was move over and TOP and solved for it which resulted in a OCnumber. Since I only get 2^16 bits to work with I need to make sure my OCnumber doesn’t go beyond that, which is done by:

OCnumber <= pow(2,16)-1

If this is true then the prescalar gets set by doing

TCCR1B_reg |= prescalers[x];

The next part of the code basically splits the up 16 bit counter value into two 8bit registers. Remember you have to right to the high byte first and then to the low because when you write to low it writes to both high byte and low byte simultaneously (this is because first writing to the high byte stores it in a temporary register.

TIMSK1_reg = 0;//0b00100010; //writing so that output compare A is set up

//enable global interrupts
 SREG_reg = Bit_7;

The mask is then turned off so that the PWM can work and then enabling global interrupts to go back to business as usual.

Here are some useful link(s) I found:

https://paritycheck.wordpress.com/2009/03/27/what-does-this-mean-volatile-unsigned-char-0x22/

My code can be found at my github.

Toggling a pin in C on the Arduino uno

It has been a while since I have posted, mainly because I have been trying to get to a stopping point, where I could just spend a few days writing. I switched over the arduino uno, mainly because of the issues I have outlined (I loose connecting to the computer as soon as the sparkfun micro gets up and running). I decided to start from scratch and work on the basics and then went up from there. So the first thing I started with was trying to blink an led on the arduino uno. The code is very similar to sparkfun micro. In order to upload my code onto arduino uno I used this link to get me started and then combined it with arduino sketch output. I ended up with the following script

avr-gcc -Os -DF_CPU=16000000UL -mmcu=atmega328p -c -o blink.o blink.c
avr-gcc -mmcu=atmega328p blink.o -o blink
avr-objcopy -O ihex -R .eeprom blink blink.hex

#flashing the Arduino:
#avrdude -V -F -C /etc/avrdude/avrdude.conf -p atmega328p -P /dev/ttyUSB0 -c stk500v1 -b 57600 -U flash:w:blink.hex
avrdude -C/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/etc/avrdude.conf -v -patmega328p -carduino -P/dev/ttyACM0 -b115200 -D -Uflash:w:blink.hex:i

I was able to compile and upload to the uno using the software above. Information from Balau’s website was instrumental in getting me started on the uno. The C code for blinking an led was:

#include <avr/io.h>
#include <util/delay.h>

int main (void)
{
 /* set PORTB for output*/
 DDRB = 0xFF;

while (1)
 {
 /* set PORTB.5 high */
 PORTB = 0x20;

_delay_ms(200);

/* set PORTB.5 low */
 PORTB = 0x00;

_delay_ms(200);
 }

return 1;
}

This allowed me to blink an led. I have stepped away from using my own header file because it became quite a pain in the neck to carry around those header files and plus I need to figure out how to link up the header files (which needs to happen as my projects get more complex). Looking at the arduino website, the led pin seems to be on what is defined as D13. After looking at the schematic , the Pin 13 is found on PB5. This can be found here:Screenshot from 2017-09-02 14:13:03

As you can see D13 or SCK is also found on PB5. Once you toggle that pin correctly you should be able to see led toggle.

 

Here is the list of the useful links I used:
https://github.com/dataplayer/Blinking-LED-on-Arduino

https://balau82.wordpress.com/2011/03/29/programming-arduino-uno-in-pure-c/

https://www.arduino.cc/en/Tutorial/Blink

My code can be found on github.

Getting up and running on SparkFun ProMicro without Arduino IDE and using C to program the ProMicro.

So this was an effort in reverse engineering. There were a lot of things that I had to strip apart in order to get this to work.  As I suggested before, in the Arduino IDE under File>Preferences there is a place to edit the preferences.txt file (you might have to get there some other way), and in there set the build.verbose and upload.verbose to True.  This allows you see the output of how the files are being compiled and built and then being uploaded onto the SparkFun ProMicro. Here is a great high level overview on how avr-gcc chain works.

The first thing I did was create 2 very different sets of code that would allow me to debug if anything failed at a certain step. The 2 sets of code I created were:

The first one was just a simple code that blinks 2 leds and this was done entirely in the arduino IDE and used all the functions that arduino provides:

/* Pro Micro Test Code
 by: Nathan Seidle
 modified by: Jim Lindblom
 SparkFun Electronics
 date: September 16, 2013
 license: Public Domain - please use this code however you'd like.
 It's provided as a learning tool.

This code is provided to show how to control the SparkFun
 ProMicro's TX and RX LEDs within a sketch. It also serves
 to explain the difference between Serial.print() and
 Serial1.print().
*/

#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<unistd.h>

//int RXLED = 17; // The RX LED has a defined Arduino pin
// The TX LED was not so lucky, we'll need to use pre-defined
// macros (TXLED1, TXLED0) to control that.
// (We could use the same macros for the RX LED too -- RXLED1,
// and RXLED0.)

void setup()
{
 //pinMode(RXLED, OUTPUT); // Set RX LED as an output
 //setting up PBO
 (*(volatile unsigned char *)(0x24)) = 0x1; //setting the 0th bit in DDRB to 1 so that it can set up for output
 (*(volatile unsigned char *)(0x25)) = 0x0; //setting the 0th bit in PortB to 0 so that it can set output low

//setting up PD5
 (*(volatile unsigned char *)(0x2A)) = 0x20; //setting the 5th bit in DDRB to 1 so that it can set up for output
 (*(volatile unsigned char *)(0x2B)) = 0x0; //setting the 5th bit in PortB to so that it can set output low

// TX LED is set as an output behind the scenes

Serial.begin(9600); //This pipes to the serial monitor
 Serial1.begin(9600); //This is the UART, pipes to sensors attached to board
}

void loop()
{
 //Serial.println("Hello world"); // Print "Hello World" to the Serial Monitor
 Serial1.println("Hello!"); // Print "Hello!" over hardware UART


 //digitalWrite(RXLED, LOW); // set the LED on
 (*(volatile unsigned char *)(0x25)) = 0x0;
 //Serial.println((*(volatile unsigned char *)(0x23)));
 
 (*(volatile unsigned char *)(0x2B)) = 0x20; //setting the 5th bit in PortB to so that it can set output high
 //TXLED0; //TX LED is not tied to a normally controlled pin

delay(1000); // wait for a second
//
 //digitalWrite(RXLED, HIGH); // set the LED off
 (*(volatile unsigned char *)(0x2B)) = 0x0; //setting the 5th bit in PortB to so that it can set output low
 
 //TXLED1;
 (*(volatile unsigned char *)(0x25)) = 0x1;
 //Serial.println((*(volatile unsigned char *)(0x23)));


 delay(1000); // wait for a second


}

 

The second one was a program that sent Pin B5 high and low as fast as it could.  This could addressed the DDRB and PortB directly with the registers and removed as much dependency on arduino as possible. The code was:

blah

#include <avr/io.h>
#include <util/delay.h>

#define BLINK_DELAY_MS 100

int main (void)
{
 /* set pin 5 of PORTB for output*/
 (*(volatile unsigned int*)(0x24)) |= (1<<(5));

while(1) {
 /* set pin 5 high to turn led on */
 (*(volatile unsigned int*)(0x25)) |= (1<<(5));
 
 /* set pin 5 low to turn led off */
 (*(volatile unsigned int*)(0x25)) &= (0<<(5));
 }
 }

I had to use avr/io and util/delay.h because those are atmel avr functions not ardunio functions. As you can see this is written in code.  I tested both the codes in Arduino and they were working. This confirmed that code base I had was working completely.

The next step was to take the output of the arduino and start breaking it apart in order to get a better understanding of what is happening.  Here is the arduino output:

 

home/ashwini/Downloads/arduino-1.8.3/arduino-builder -dump-prefs -logger=machine -hardware /home/ashwini/Downloads/arduino-1.8.3/hardware -hardware /home/ashwini/.arduino15/packages -tools /home/ashwini/Downloads/arduino-1.8.3/tools-builder -tools /home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr -tools /home/ashwini/.arduino15/packages -built-in-libraries /home/ashwini/Downloads/arduino-1.8.3/libraries -libraries /home/ashwini/Arduino/libraries -fqbn=SparkFun:avr:promicro:cpu=8MHzatmega32U4 -ide-version=10803 -build-path /tmp/arduino_build_569596 -warnings=all -build-cache /tmp/arduino_cache_732047 -prefs=build.warn_data_percentage=75 -verbose /home/ashwini/Arduino/blinky_c_no_serial_for_avrgcc/blinky_c_no_serial_for_avrgcc.ino

/home/ashwini/Downloads/arduino-1.8.3/arduino-builder -compile -logger=machine -hardware /home/ashwini/Downloads/arduino-1.8.3/hardware -hardware /home/ashwini/.arduino15/packages -tools /home/ashwini/Downloads/arduino-1.8.3/tools-builder -tools /home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr -tools /home/ashwini/.arduino15/packages -built-in-libraries /home/ashwini/Downloads/arduino-1.8.3/libraries -libraries /home/ashwini/Arduino/libraries -fqbn=SparkFun:avr:promicro:cpu=8MHzatmega32U4 -ide-version=10803 -build-path /tmp/arduino_build_569596 -warnings=all -build-cache /tmp/arduino_cache_732047 -prefs=build.warn_data_percentage=75 -verbose /home/ashwini/Arduino/blinky_c_no_serial_for_avrgcc/blinky_c_no_serial_for_avrgcc.ino

Using board 'promicro' from platform in folder: /home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6

Using core 'arduino' from platform in folder: /home/ashwini/Downloads/arduino-1.8.3/hardware/arduino/avr

Detecting libraries used...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/dev/null"

Generating function prototypes...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp"

"/home/ashwini/Downloads/arduino-1.8.3/tools-builder/ctags/5.8-arduino11/ctags" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives "/tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp"

Compiling sketch...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp.o"

Compiling libraries...

Compiling core...

Using precompiled core

Linking everything together...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-gcc" -Wall -Wextra -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -mmcu=atmega32u4 -o "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.elf" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp.o" "/tmp/arduino_build_569596/../arduino_cache_732047/core/core_SparkFun_avr_promicro_cpu_8MHzatmega32U4_1e6cbd299ffad185359e932e3440541e.a" "-L/tmp/arduino_build_569596" -lm

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-objcopy" -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.elf" "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.eep"

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-objcopy" -O ihex -R .eeprom "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.elf" "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.hex"

Sketch uses 230 bytes (0%) of program storage space. Maximum is 28672 bytes.
Global variables use 0 bytes (0%) of dynamic memory, leaving 2560 bytes for local variables. Maximum is 2560 bytes.

/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avrdude -C/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/avrdude.conf -v -patmega32u4 -cavr109 -P/dev/ttyACM0 -b57600 -D -Uflash:w:/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.hex:i

These were the set of commands used and yes it seems very daunting, but there is a method to madness that you can parse away, hopefully this will also help people get a better understanding of how to reverse engineer things (even though I am pretty bad at it). The thing to note is that these commands can be directly ran on the Linux command line. The first step for me was to get rid of the everything that depends on ino files and look for files that are c/c++ related (files with endings of c and cpp). The first thing to note is that the first 2 lines convert the into a tmp directory and build a cpp file and those 2 lines are:

home/ashwini/Downloads/arduino-1.8.3/arduino-builder -dump-prefs -logger=machine -hardware /home/ashwini/Downloads/arduino-1.8.3/hardware -hardware /home/ashwini/.arduino15/packages -tools /home/ashwini/Downloads/arduino-1.8.3/tools-builder -tools /home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr -tools /home/ashwini/.arduino15/packages -built-in-libraries /home/ashwini/Downloads/arduino-1.8.3/libraries -libraries /home/ashwini/Arduino/libraries -fqbn=SparkFun:avr:promicro:cpu=8MHzatmega32U4 -ide-version=10803 -build-path /tmp/arduino_build_569596 -warnings=all -build-cache /tmp/arduino_cache_732047 -prefs=build.warn_data_percentage=75 -verbose /home/ashwini/Arduino/blinky_c_no_serial_for_avrgcc/blinky_c_no_serial_for_avrgcc.ino

/home/ashwini/Downloads/arduino-1.8.3/arduino-builder -compile -logger=machine -hardware /home/ashwini/Downloads/arduino-1.8.3/hardware -hardware /home/ashwini/.arduino15/packages -tools /home/ashwini/Downloads/arduino-1.8.3/tools-builder -tools /home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr -tools /home/ashwini/.arduino15/packages -built-in-libraries /home/ashwini/Downloads/arduino-1.8.3/libraries -libraries /home/ashwini/Arduino/libraries -fqbn=SparkFun:avr:promicro:cpu=8MHzatmega32U4 -ide-version=10803 -build-path /tmp/arduino_build_569596 -warnings=all -build-cache /tmp/arduino_cache_732047 -prefs=build.warn_data_percentage=75 -verbose /home/ashwini/Arduino/blinky_c_no_serial_for_avrgcc/blinky_c_no_serial_for_avrgcc.ino

I at this point realized that the c/cpp files are stored in

/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp

because it is the same name as ino file also. I then removed the first 2 lines and commented out lines for generating function prototypes just to see what it would do and it seemed to have no impact on the build process. Note: Right before running the avrdude command to upload I had to always reset the MCU in order for the upload to be successful. In order to make sure I was doing everything right, I would first use the led blinking script to make sure things are working and then just compile the script that doesn’t have any dependency on Arduino and then try to upload everything one line at a time on the cmdline. So the new script now became:

 

#removed all the dependency of the arduino core libraries and removed all appearance of ino files, even though ino files have not been removed

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/dev/null"

#Generating function prototypes...

#"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp"

#"/home/ashwini/Downloads/arduino-1.8.3/tools-builder/ctags/5.8-arduino11/ctags" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives "/tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp"

#Compiling sketch...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp.o"

#Compiling libraries...

#Compiling core...

#Using precompiled core

#Linking everything together...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-gcc" -Wall -Wextra -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -mmcu=atmega32u4 -o "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.elf" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp.o" "/tmp/arduino_build_569596/../arduino_cache_732047/core/core_SparkFun_avr_promicro_cpu_8MHzatmega32U4_1e6cbd299ffad185359e932e3440541e.a" "-L/tmp/arduino_build_569596" -lm

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-objcopy" -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.elf" "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.eep"

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-objcopy" -O ihex -R .eeprom "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.elf" "/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.hex"

#Sketch uses 230 bytes (0%) of program storage space. Maximum is 28672 bytes.
#Global variables use 0 bytes (0%) of dynamic memory, leaving 2560 bytes for local variables. Maximum is 2560 bytes.

/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avrdude -C/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/avrdude.conf -v -patmega32u4 -cavr109 -P/dev/ttyACM0 -b57600 -D -Uflash:w:/tmp/arduino_build_569596/blinky_c_no_serial_for_avrgcc.ino.hex:i

#this still turns out to be succesfull

As you can see the ino depedency is gone. The ‘#’ in these scripts represent a comment. I played around with the ctags but that seemed to very little impact on my code so I removed all ctags information with the following command:

rm /tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp

and it didn’t seem to have a large impact. The next step was to completely remove all notion of using tmp files and copy the c file that I created which I wrote as an ino file and move it to my git repo so I could play with the file. The file I created was called led.c (yea I know I am using confusing nomenclature, but at after a certain point at night everything starts to blur together). The next code got rid of all dependency on the tmp directory and it looked like this

#replacing all tmp directory calls to be in the git repo
"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.cpp" -o "/dev/null"

#Generating function prototypes...

#"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/tmp/arduino_build_569596/sketch/blinky_c_no_serial_for_avrgcc.ino.cpp" -o "/tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp"

#"/home/ashwini/Downloads/arduino-1.8.3/tools-builder/ctags/5.8-arduino11/ctags" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives "/tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp"

rm /tmp/arduino_build_569596/preproc/ctags_target_for_gcc_minus_e.cpp

#Compiling sketch...

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-g++" -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.cpp" -o "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.cpp.o"




#Compiling libraries...

#Compiling core...

#Using precompiled core

#Linking everything together...

rm /tmp/arduino_build_569596/../arduino_cache_732047/core/core_SparkFun_avr_promicro_cpu_8MHzatmega32U4_1e6cbd299ffad185359e932e3440541e.a

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-gcc" -Wall -Wextra -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -mmcu=atmega32u4 -o "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.elf" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.cpp.o" "-L/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide" -lm

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-objcopy" -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.elf" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.eep"

"/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avr-objcopy" -O ihex -R .eeprom "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.elf" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.hex"

#Sketch uses 230 bytes (0%) of program storage space. Maximum is 28672 bytes.
#Global variables use 0 bytes (0%) of dynamic memory, leaving 2560 bytes for local variables. Maximum is 2560 bytes.

read -p "Reset the sparkfun micro and then press any key to flash the arduino"

/home/ashwini/Downloads/arduino-1.8.3/hardware/tools/avr/bin/avrdude -C/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/avrdude.conf -v -patmega32u4 -cavr109 -P/dev/ttyACM0 -b57600 -D -Uflash:w:/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.hex:i

#this works also I was able to replace all the extra calls to the tmp directory and do it all in c

This was the biggest clue I was going down the right path, because I used a file called led.c:

#include <avr/io.h>
#include <util/delay.h>

#define BLINK_DELAY_MS 100

int main(void);
int main (void)
{
 /* set pin 5 of PORTB for output*/
 (*(volatile unsigned int*)(0x24)) |= (1<<(5));

while(1) {
 /* set pin 5 high to turn led on */
 (*(volatile unsigned int*)(0x25)) |= (1<<(5));

/* set pin 5 low to turn led off */
 (*(volatile unsigned int*)(0x25)) &= (0<<(5));
 }
}

and since this worked I knew everything was on the right path and remember I always verify by going back to a known state which is using the scrip that blinks to LEDs using the arduino and then I can always come back to toggling the PB5  when I upload my code so that really helped. The next step was to remove all commands being executed from the arduino directory and use the base avr-gcc available on my linux computer and the script was:

#replacing all tmp directory calls to be in the git repo
avr-gcc -c -g -Os -w -fno-exceptions -ffunction-sections -fdata-sections -flto -w -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.c" -o "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/log.txt"

#Compiling sketch...

avr-gcc -c -g -Os -Wall -Wextra -fno-exceptions -ffunction-sections -fdata-sections -MMD -flto -mmcu=atmega32u4 -DF_CPU=8000000L -DARDUINO=10803 -DARDUINO_AVR_PROMICRO -DARDUINO_ARCH_AVR -DUSB_VID=0x1b4f -DUSB_PID=0x9204 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="SparkFun Pro Micro"' "-I/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/variants/promicro" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.c" -o "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.o"




#Compiling libraries...

#Compiling core...

#Using precompiled core

#Linking everything together...

avr-gcc -Wall -Wextra -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -mmcu=atmega32u4 -o "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.elf" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.o" "-L/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide" -lm

avr-objcopy -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.elf" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.eep"

avr-objcopy -O ihex -R .eeprom "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.elf" "/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.hex"

#Sketch uses 230 bytes (0%) of program storage space. Maximum is 28672 bytes.
#Global variables use 0 bytes (0%) of dynamic memory, leaving 2560 bytes for local variables. Maximum is 2560 bytes.

read -p "Reset the sparkfun micro and then press any key to flash the arduino"

avrdude -C/home/ashwini/.arduino15/packages/SparkFun/hardware/avr/1.1.6/avrdude.conf -v -patmega32u4 -cavr109 -P/dev/ttyACM0 -b57600 -D -Uflash:w:/home/ashwini/git-repos/arduino/led_blink_without_arduino_ide/led.hex:i

#this works also I was able to replace all the extra calls to the tmp directory and do it all in c, removed all c++ specific flags. Now removed all the arduino specific gcc calls and now using system avr-gcc and avr-objcopy and avrdude

The read part I have before avr dude allows me to reset the pro micro before I flash it using avr dude. I completely removed all arduino related dependency.  I still do need to point to the ProMicro configuration file in order to use the header files that are available and to properly upload the executable file.  This allowed me to completely run without the arduino IDE. The next step is to figure out how all the functions work. I looked in ELF (Executable and Linkable Format) files which are extremely useful in these conversations and i found this link and this link, which were extremely useful in explaining how an elf file is created and made. This link was also useful. I was still in the process of decoupling all the flags but

-ffunction-sections -fdata-sections

are used so that every function I create get goes in a seperate section and each piece of data gets a seperate section. The -D flag are macros, which can also be set in a #define as shown in this link. Here is another link that explain those compiler optimizations.

UPDATE:

As I was playing around I have noticed that the manual version that I created to compile and upload the to sparkfun causes an issue.  The reason for this because the arduino has a main.ccp file which runs code called USBDevice.attach() which is relevant for arduino leonard (or the sparkfun pro micro for that matter). This is important because it allows the usb device to stay attached after the hex file is uploaded to mcu. If this is not in your file I saw /dev/ttyACM0 get attached for 5-15 secs and then you see the device disappear. There is a usb attach function that keeps it attached but for now I think that will be too confusing for me so I have decided to switch to Arduino uno to make my life a little easier. I will let you know when I come back to sparkfun pro micro.

As always here is the link to the github.

Here is my list of useful links:

https://arduino.stackexchange.com/questions/9064/trying-to-track-down-macro-or-define-values-possibly-in-arduino-preprocessor

https://balau82.wordpress.com/2011/03/29/programming-arduino-uno-in-pure-c/

http://www.nongnu.org/avrdude/user-manual/avrdude_10.htmlhttps://www.arduino.cc/en/Hacking/Bootloader

https://github.com/arduino/Arduino/wiki/Build-Process

https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification

https://github.com/arduino/Arduino/wiki/Arduino-Hardware-Cores-migration-guide-from-1.0-to-1.6

https://forum.arduino.cc/index.php?topic=158701.0

https://www.mankier.com/1/arduino-ctagshttps://stackoverflow.com/questions/20177113/how-to-disable-gcc-warning-cc1-warning-command-line-option-std-c11-is-va

https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Optimize-Options

https://arduino.stackexchange.com/questions/9064/trying-to-track-down-macro-or-define-values-possibly-in-arduino-preprocessor

https://groups.google.com/forum/#!topic/fa.freebsd.current/RUZnbnYykQo

ELF information:

http://www.linuxjournal.com/article/1059?page=0,1

http://www.cirosantilli.com/elf-hello-world/

http://www.linuxjournal.com/article/1060

https://linux-audit.com/elf-binaries-on-linux-understanding-and-analysis/https://groups.google.com/forum/#!topic/fa.freebsd.current/RUZnbnYykQo