ios - CVImageBuffer comes back with extra column padding. How do I crop it? -

i have cvimagebuffer comes recorded height of 640px , width of 852px. bytes per row 3456. you'll notice 3456/852px != 4 (it's 4.05). after inspection, 864 width makes bytesperrow/width = 4.0. so, seems there 12px on each row (padded on right). i'm assuming because these buffers optimized multiple image not have.

when render out buffer in opengl looks terrible (see below). noticed pattern repeats every 71px, makes sense because if there 12px (852/12px = 71). so, 12 pixels seem causing problem.

how rid of these pixels , use data read opengl es? or rather, how read opengl es skipping these pixels on each row?

it's pretty common images used high speed image processing algorithms have padding @ end of each line image has pixel or byte size multiple of 4, 8, 16, 32, , on. makes easier optimize algorithms speed in combination simd instruction sets sse on x86 or neon on arm. in case padding 12 pixels, means apple seems optimize algorithms processing 32 pixels per line; 852 not dividable 32, 864 is, hence lines padded 12 pixels maintain 32 pixel alignment. correct technical terms size , stride size or in case of images, width , stride width. width amount of actual pixel data per line, stride width real line size, including pixel data , optional padding @ end of line.

standard opengl allows load textures stride width bigger actual texture width. achieved setting glpixelstore parameter gl_pack_row_length accordingly. note "stride padding skip" implemented within cpu part of driver, no operation performed on gpu, in fact driver removing padding before uploading data gpu. opengl es designed run on embedded devices may have limited cpu resources available, option removed opengl es keep driver development simple, weak embedded cpus. leaves 4 options deal problem:

1. preprocess texture remove padding using c copy loop, skips pixels @ end of each line. implementation rather slow easy implement.

2. preprocess texture in case of option (1), use compiler simd macros make use of neon instructions. 2 times faster option (1) it's harder implement , you'll need knowledge neon instructions , how use them achieve goal.

3. preprocess textures in case of option (2), use pure assembly implementation. 3 times faster option (2), 6 times faster option (1) it's lot harder implement, since you'll need knowledge arm assembly programming + neon instructions.

4. load texture padding , adjust texture coordinates opengl make ignore padding pixels. depending on how complex texture mapping is, might easy implement, it's faster other option above , downside waste little bit more texture memory on gpu.

i know little arm assembly programming , less neon instructions, cannot options (2) , (3). show implementation option (1), however, i'm afraid might slow purpose. leaves last option have been using myself plenty of times in past.

we declare 3 variables: width, height, , stride width.

glsizei width = 852; glsizei height = 640; glsizei stridewidth = 864; 

when load texture data (assuming rawdata points raw image bytes), pretend stridewidth "real width":

glteximage2d(gl_texture_2d, 0, gl_rgb,      stridewidth, height, 0, gl_rgb, gl_unsigned_byte, rawdata); 

texture coordinates in opengl normalized, means lower left corner (0.0f, 0.0f) , upper right corner (1.0f, 1.0f), regardless pixel size texture has. these 2 values called (x, y), not confuse them vertex coordinates, called (s, t) instead.

to make opengl cut off padding pixels, need adjust s-coordinates factor, let's call spcf (stride padding cut factor), calculate following way:

float spcf = (float)width / stridewidth; 

so instead of texture coordinate (0.35f, 0.6f), use (0.35f * spcf, 0.6f). of course shouldn't perform calculation once per rendered frame. instead should copy original texture coordinates, adjust s-coordinates once spcf , use these adjusted coordinates when rendering frames. if ever reload texture in future , spcf has changed, repeat adjustment process. in case width equals stridewidth, algorithm works well, in case spcf 1.0f , won't alter s-coordinates @ all, correct, since there no padding cut off.

the downside of trick texture need 2.4% more memory in case otherwise necessary, means texture upload via glteximage2d 2.4% slower. guess acceptable , still faster of other cpu intensive options above.