C programs for motion-compensated image sequence compression

(1)

C Programs for Motion-Compensated

Image Sequence Compression

by

Ro bert

J. Moor head II

and

Sarah A. Rajala

Center ror Cornmu'nicanions and Signal Processing

Dep arumerrt of Electrical and Computer Engineering

North Carolina State Universjty

July

1985

(2)

results in CCSP- TR-85/12. Two of the programs, grncs25i.c and gmcs27i.c, are appended

in full. The rest are indicated by the changes made to gmcszfii.c and gmcs27i.c. The

CNL"X utility program"cliff" was used to obtain the changes.

program motion prediction order of entropy noise

name scheme transmission suppression

gmcs25i.c spatial 1 yes

grncs25j .c spatial 0 yes

gmcs25s.c spatial 1 no

grncs27i.c P4~IT 1 yes

gmcs27j.c

P:\..vIT

0 yes

gmcs27s.c P4~vfT 1 no

gmcs30i.c mixed 1 yes

grncs30s.c mixed 1 no

As previously stated gmcs25i.c and grncs27i.c are included herein in full.

dif25i27i indicates the difference in using spatial prediction and P A.\1T prediction.

dif25ij indicates the differences in using zeroth-order entropy transmission and

first-order entropy transmission.

dif25is indicates the differences in the code used on real sequences (grncs25i.c) and

the synthetic sequences (gmcs25s.c). The major differences are no noise suppression In

A

gmcs25s.c and part of the d field is printed out in gmcs25s.c.

dif27ij indicates the differences in using zeroth-order entropy transmission and

(3)

2

dif27is is like dif25is, but is for P

Alvl'T

motion prediction.

dif27i30i and dif27s30s contain the additions to the P iUJT motion prediction

(4)

/*

The interframe differences are clipped via threshold(). The data is

histogrammed. The values to be sent are quantized. The entropy of

the pel difference values is calculated. The addressing requirements

are calculated. The maximum motion vector components and their locations

are determined. */

/* This simulation is derived from an algorithm developed by Netravali and

Robbins at Bell Labs. Three papers should be noted: "Motion-Compensated

Television Coding: Part I," BSTJ, March 1979, pp. 631-670; "Interframe

Coding Using Movement Compensation," ICC, June 1979, pp. 23.4.1-23.4.5; and

"Spatial Subsampling in Motion-Compensated Television Coders," BSTJ, Oct.

1982, pp. 1897-1898. We are seeking the algorithm which will give the

minimum transmitted bit rate. */

/*

This simulation version has the following unique:

1) the motion vector is maintained as a real number.

2) the displaced intensity prediction *is* interpolated.

3) the previous motion vector is the NW one.

4) the motion vector is updated using real multiplication.

5) pvalue is calculated using interpolation.

6) intraframe noise suppression (isolated point rejection).

7) mv update is product.

8) dfd[j] is updated after mv is updated.

9) mv* is updated dependent on dfd[j] only.

10) dfd[j] is interpolated in iterative mv-update loop.

11) predictor switches between CR and

Me

dependent on dfdsum/fdsum.

12) only two mv* correction choices (OK or iterate). */

quanlevs,

/*

# of columns in frame */

/* # of rows in frame */

/*

maximum quantization levels */

/* maximum number of iterations */

/* number of quantization levels (must be odd)c

Includes positive and negative levels */

/* value at end of each quantization range */

/* quantization value (level) for each

quanti-zation range */

/* length of code word at quantization level */

/* difference frame for isolated pt. reject */

<stdio.h> <math.h>

"/usr/local/dsp/dsp2.h"

NeOL 448

NROW 282

QLEV 40

MAXI 40

quanept[QLEV] , quanval[QLEV] , quanlen[QLEV] , dframe[NCOL*NROW]; int

fJ'include If-include If-include If-define fFdefine fFdefine fJ'define 1

2 3 4 5

6

(5)

Aug 15 15:28 1985 gmcs25i.c Page 2

46

47 main()

48 {

49 char fname1[40], /* the previously transmitted frame */

50 fname2[40], /* the next frame to be transmitted */

51 fname3[40], /* the predicted

&

uncorrected output frame */

&

corrected output frame */

53 inpre[30], /* the prefix common to every input frame */

54 outprel[31], /* the prefix common to every fname3 */

55 outpre2[31]; /* the prefix common to every fname4 */

S6 struct data fl,*pfl; /* the numbers correspond to numbers above */

57 struct data f2,*pf2;

S9 struct data f4,*pf4;

60 register int i,j; /* indexing variables */

61 int k,

62 err,

63 nrows, /* the number of rows 10 the source frame */

64 ncols, /* the number of cols 1n the source frame */

6S npels, /* the number of pels 1n the source frame */

66 peldif, /* interframe difference used in noise red */

67 (*threshold)(), /* pointer to a threshold function */

68 correction, /* difference between correct value and

69 predicted value */

70 prevdif, /* previous difference of differences */

71 dlevel, /* quantization level of dif of prevdifs-*/

72 level, /* quantization level of correction value */

73 runlength, /* number of skipped zero values */

74 value, /* the corrected value in the predicted frame */

75 pvalue, /* displaced predicted value */

76 imvv, /* vertical component of the motion vector */

77 imvh, /* horizontal component of the motion vector */

78 nclips, /* number of times nimvv/nimvh was clipped */

79 resetd, /* number of times mv* is reset each frame */

80 fracintv, /* vertical interpolation fraction */

81 fracinth, /* horizontal interpolation fraction */

82 ij, /* present pel address */

83 IJmaxv. /* location of maximum vertical displacement */

84 ijmaxh, /* location of maximum horizontal displace */

85 ijdis,

/*

displaced pel address in previous frame */

86 ijdisv, /* nearest vertical location to ijdis */

87 ijdish, /* nearest horizontal location to ijdis */

88 signum, /* threshold on "sign-of-value" for update */

89 motthres, /* motion threshold

*/

90 fdsum, /* sum of frame differences */

91 dfdsum, /* sum of displaced frame differences */

92 dfd[NCOL], /* the DFOs in present line */

93 odfd(NCOL+2], /* the DFOs in previous line */

94 fdif[NCOL+2], /* the FOs in previous line */

95 num, /* number of frames to be processed */

96 insuf, /* the integer part of every image */

97 11, /* frame indexing variable */

98 valhis[QLEV], /* used to histogram values transmitted */

99 dvalhis[QLEV], /* used to histogram dif values transmitted */

(6)

mvv, mvh, mvva[NCOL+2], mvha[NCOL+2], omvva[NCOL+2], omvha[NCOL+2], maxmvv, maxmvh, addsum, pixsum, entropy, meansqer, s n; hardcopy[10], frameout[10J, terminal[10]; 101 102 103 104 105 106 107 108 109 110 III 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 float char qbitsum, /* ebitsum, /* infobits, /* infobits2, /* addrbits, /* bpf, /* predpell, /* predpe12, /* itermax, /* iter, /* iteration[MAXI],/* itertot[MAXI], /* gradnum, /* noconv, /* diff, /* sqerr; /* updatev, /* updateh, /* edif, /* ldif, /* xgradsum, /* ygradsum, /* zgradsum, /* xgradmax, /* ygradmax, /* zgradmax, /* epsilon, /* cepsilon, /* ncepsilon, /* fdisv, /* fdish, /* /* /* /* /* /* /* /*

/*

/* /* /*

/*

total number of quantizer bits transmitted ~I

total number of entropy bits in a frame *1

total number of diE-of-diE bits */

total number of st. diff bits (approx.) */

total number of address bits */

total bits per frame (info + addr) */

number of predictable pels in each frame */

number of predictable pels in each frame ~/

the maximum number of iterations «MAXI) */

which iteration is being performed

*/

how many times we've iterated this far

*/

total number of iterations in sequence */

total number of gradients in a frame */

incremented each time we don't converge */

frame difference (error) */

sum of squared errors in a frame */

the vertical update component */

the horizontal update component */

element difference */

line difference */

total square of edge gradients per frame */

total square of line gradients per frame

*/

total square of cross gradients per frame */

maximum edge gradient in a frame */

maximum line gradient in a frame */

maximum cross gradient in a frame */

converge coefficient */

maximum update value */

additive inverse of maximum update value */

fractional part of vertical displacement */

fractional part of horizontal displacement */

vertical component of the motion vector */

horizontal component of the motion vector */

vertical mv's on present line */

horizontal mvts on present line */

vertical mvts on previous line */

horizontal mv's on previous line */

maximum vertical displacement */

maximum horizontal displacement */

number of runlengths sent

*/

total number of pels scanned */

p*log(p) of quantization words */

mean square error in a frame */

signal to noise ratio in a frame */

"verbose" ('v' ••• ) or "short" */

"yes" ('y' ••• ) or "no" */

(7)

Aug 15 15:28 1985 gmcs25i.c Page 4

149 150 151 152 153 154

ISS

156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203

/,'~,,, _{set file pointers} _*/

pi1

=

&f1;

pf2

=

&f2; pf3 &£3; pf4

=

&f4;

/,'~,.. _{get file names} ..,':

/

err = 0;

k

=

getline(fnamel,"name of previous frame:O);

printf("Ohe name of the previous frame: %sO,fnamel);

if (k<O) err =1;

k = getline(inpre,"prefix common to every input frame:O);

printf("The prefix common to every input frame: %sO,inpre);

if (k<O) err =9;

k = getline(outprel,"prefix common to every outputl frame:O);

printf("The prefix common to every output1 frame: %sO,outprel);

if (k<O) err =10;

k = getline(outpre2~"prefix common to every output2 frame:O);

printf("The prefix common to every output2 frame: %sO,outpre2);

if (k<O) err =11;

fprintf(stderr,"suffix of first frame to be transmitted (integer):O); scanf("%d",&insuf);

printf("The suffix of the first frame to transmit: %2dO,insuf);

fprintf(stderr,"the number of frames to process (integer):O); scanf("%d",&num);

printf("The number of frames to process was: %2dO,num);

sprintf(fname2,"%s%d.hd",inpre,insuf); sprintf(fname3,"%s%d.hd",outprel,insuf); sprintf(fname4,"%s%d.hd",outpre2,insuf);

fprintf(stderr,"What is the convergence coefficient?O); scanf("%f",&epsilon);

printf(ttThe convergence coefficient was: %.6fO,epsilon);

fprintf(stderr,"What is the maximum update value?O); scanf("%f",&cepsilon);

printf{ltThe maximum update value was: %.6fO,cepsilon);

ncepsilon = -cepsilon;

fprintf(stderr,"What is the update-displacement threshold?O); scanf("%d",&motthres);

printf("The update-displacement threshold was: %ldO,motthres);

fprintf{stderr,"What is the signum threshold?O); scanf{"%d",&signum);

printf("The signum threshold was: %ldO,signum);

fprintf(stderr,"What is the maximum number of iterations?O); scanf("%d",&itermax);

printf("The maximum number of iterations was: %2dO,itermax);

fprintf(stderr~"Ooyou want verbose or short hardcopy output?O);

scanf("%s",hardcopy);

printf("The output was made: %sO,hardcopy);

fprintf{stderr,"do you want the output frames saved?O); scanf("%s",frameout);

printf{tlWere the output frames saved? %sO,frameout);

(8)

read in first two frames

*/

k = pixin(pEl,fnamel,"silent");

if (k<O) err =2;

k = pixin(pf2,fname2,"silentft

) ;

if (k<O) err =4;

if (err>O) {

fprintf(stderr,"err

=

%Id; error 1n getting a file.O,err);

exit(-l); 204

205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259

/*

scanE("%s",terminal);

printf(ltThe terminal session was:

set parameters for quantizer from a quanfile

*/

setpar();

select threshold function

*/

selthr(&threshold);

}

initialize various values

*/

nrows = pfl->nline;

ncols = pfl->npix;

npels

=

nrows*ncols;

printf("The number of pels in each frame was:

printE(" 14 %s: 0,fname2);

makehead(pf3,pfl,COPYHEAD); makehead(pf4,pfl,COPYHEAD);

mvv = 0.0;

mvh = 0.0;

maxmvv = 0.0;

maxmvh = 0.0;

ijrnaxv

=

0;

ijmaxh = 0;

dfd[O]

=

0;

prevdi£

=

0;

runlength = 0;

addsurn = 0.0;

infobits

=

0;

infobits2 = 0;

addrbits = 0;

Ear (i=O; i<QLEV; i++)

I

valhis[i]

=

0;

dvalhis[i]

=

0;

}

for (i=O; i<=NCOL; i++)

addhis[i] = 0;

for (i=O; i<MAXI; i++)

iteration[i]

=

0;

for (i=O; i<MAXI; i++)

itertot[i] = 0;

noconv = 0;

nclips

=

0;

resetd = 0;

gradnum = 0;

xgradsum = 0.0;

ygradsum

=

0.0;

%sO,terminal);

(9)

zgradsum = 0.0; xgradmax = 0.0;

ygradmax = 0.0;

zgradmax = 0.0;

Aug 15 15:28 1985 gmcs25i.c Page 6

260 261 262 263 264

265 /* reduce interframe noise */

266 if (terminal[O] == 'v')

267 fprintf(stderr,"start noise suppressionO);

268 for (i=O; i<ncols*nrows; i++) {

269 peldif = (pf2->un.aryc[i]

&

00377)

270 - (pf1->unoaryc[i] & 00377);

271 dframe[i]

=

(*threshold)(peldif);

272

273 /* now do intraframe noise suppression (isolated pt. rejection)

274 in scanning order, "ignoring" the 4 corners of the image. "":/

275 /* first row a special case; skip first and last pel */

276 pf2->un.aryc[0] = (pfl->un.aryc[O] & 00377) + dframe[O];

277 for (j=l; j<ncols-l; j++)

278 if(ldframe[j-1]

&&

!dframe[j+l] && !dframe[j+ncols]) {

279 if «dframe[j]>S) && (terminal[O] == 'v')

280 fprintf(stderr,"ij=%6d,dif=%4dO,j,dframe[j]);

281 pf2->un.aryc[j] = (pfl->un.aryc[j] &00377);

282 }

283 else

284 pf2->un.aryc[j] = (pfl->un.aryc[j] & 00377) + dframe[j];

285 j

=

ncols~l;

286 pf2->unearyc[j] = (pf1->un.aryc[j]

&

00377) + dframe[j];

287 /* continue through next-to-last row */

288 for (i=l; i<nrows-l; i++) {

289 ij = i*ncols;

290 if(!dframe[ij-ncols] && Idframe[ij+1]

&&

!dframe[ij+ncols]) {

291 if «dframe[ij]>S) && (terminal[O] == 'v')

292 fprintf (s tderr," i j=%6d, dif=%4dO, i j ,dframe[i j]);

293 pf2~>un.aryc[ij] = (pfl->un.aryc[ij] & 00377);

294 }

295 else

296 pf2->un.aryc[ij] = (pfl->un.aryc[ij] & 00377) + dframe[ij];

297 for (j=l; j<ncols-l; j++) {

298 ij = i*ncols+j;

299 if(dframe[ij-l]==O)&&(dframe[ij-ncols]==O)&&

300 (dframe[ij+1]==O)&&(dframe[ij+ncols]==0» {

301 if «dframe[ij]>S)

&&

(terminat[O] ==

'v'»

302 fprintf(stderr,"ij=%6d,dif=%4dO,ij,dframe[ij]);

303 pf2->un.aryc[ij] = (pfl->un.aryc[ij] & 00377);

304 }

305 else

306 pf2->un.aryc[ij] = (pfl->un.aryc[ij]

&

00377) + dframe[ij];

307 }

308 1J = i*ncols+ncols-l;

309 if(!dframe[ij-1] && !dframe[ij-ncols] && !dframe[ij+ncols]) {

310 if «dframe[ij]>S) && (terminal[O] == 'v')

311 fprintf(stderr,"ij=%6d,dif=%4dO,ij,dframe[ij]);

312 pf2->un.aryc[ij] = (pfl->un.aryc[ij] &00377);

313 }

314 else

(10)

316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333

}

/* do last row similar to first row */

i

=

nrows-1;

ij = i*ncols;

pf2->un.aryc[ij] = (pf1->un.aryc[ij] &00377) + dframe[ij];

for (j=l; j<ncols-l; j++) {

i j = i;':ncol s+j ;

if(!dframe[ij-l] && ldframe[ij-ncols] && !dframe[ij+l]) {

if «dframe[ij]>S) && (terminat[D] ==

'v'»

fprintf (stderr, .,i j=%6d, di f=%4dO, i j, dfr ameli j ] ) ;

pf2->un.aryc[ij] = (pfl->un.aryc[ij]

&

00377);

} else

pf2->un.aryc[ij]

=

(pfl->un.aryc[ij] &00377) + dframe[ij];

ij nrows*ncols-l;

(11)

Aug 15 15:28 1985 gmcs2Si.c Page 8

334 /* recurse and predict

*/

335 if (terminal[O] == 'v')

336 fprintf(stderr,"start recursing and predictingO);

337 for(i=O; i<nrows; i++) {

338

/*

load the DFDs and FOs from previous line

*/

339 if (i > 0)

340 for(j=O; j<ncols; j++) {

341 odfd[j+1] = dfd[j];

342 fdif[j+l] = (pf4->un.aryc[(i-1)*ncols+j]

&

00377)

343 - (pfl->un.aryc[(i-l)*ncols+j]

&

00377);

344 omvva[j+1] = mvva[j];

345 omvha[j+1]

=

mvha[j];

346 }

347 else /* once we start looking at sequences, this might disappear?

*/

349 odfd[j+l] = 512;

350 fdif[j+1] = 0;

351 omvva[j+l] = 0;

352 omvha[j+l] = 0;

353 }

354 odfd[O]

=

(odfd[1]+odfd[2]»>1;

355 odfd[ncols+l]

=

(odfd[ncols-l]+odfd[ncols]»>l;

356 fdif[O] = (fdif[l]+fdif[2]»>1;

357 fdif[ncols+1] = (fdif[ncols-1]+fdif[ncols]»>1;

358 omvva[O]

=

omvva[l];

359 omvva[ncols+l] = omvva[ncols];

360 omvha[O]

=

omvha(l];

361 omvha[ncols+l]

=

omvha[ncols];

362

363 /* predict and transmit values on this line */

365 ij = i*ncols + j;

366 dfdsum = abs(odfd[j])+abs(odfd[j+l])+abs(odfd[j+2]);

367 fdsum = abs(fdif[j])+abs(fdif[j+l])+abs(fdif[j+2]);

368 if (j == 0)

369 mvv = omvva[j+l]; /* None */

370 else

371 mvv = omvva[j]; /* NW one */

372 if (j

==

0)

373 mvh = omvha[j+l]; /* None */

374 else

375 mvh = omvha[j]; /* NW one */

376 imvv

=

«mvv>O) ? (int)(mvv+OeS) (int)(mvv-O.5»;

377 imvh

=

«mvh>O) ? (int)(mvh+0.5) : (int)(mvh-005));

378

379 /* let's interpolate the intensity values

*/

380 fdisv = imvv - mvv;

381 fdish = imvh ~ mvh;

382 /* if the displacement is outside the frame area,

383 clip to one row/column from edge and set the

384 fractional values to zero. We zero both fractional

385 values if we clip, because of indexing scheme.

*/

386 if (i+imvv < 0) {

387 imvv

=

-i;

(12)

389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444

fdish = 0.0;

}

if (i+imvv > nrows-1)

imvv = nrows-i-l;

fdisv = 0.0;

fdish = 0.0;

}

if (imvh > j) {

imvh = j;

fdisv = 0.0;

fdish = 0.0;

}

if (imvh < j-neols+l) {

imvh = j-neols+l;

fdisv = 0.0;

fdish = 0.0;

}

ijdis

=

(i+imvv)*ncols + j-imvh;

if (imvv > mvv) {

ijdisv

=

ijdis - neals;

fracintv = fdisv * «pfl->un.aryc[ijdisv] &00377)

- (pfl->un.aryc[ijdis] &00377»;

}

else {

ijdisv = ijdis + neals;

fracintv = fdisv

_*

«pfl->un.aryc[ijdis] & 00377)

-

(pf1->un.arye[ijdisv] & 00377»;

}

if (imvh > mvh) {

ijdish = ijdis + 1;

fracinth = fdish * «pfl->un.aryc[ijdish] & 00377)

-

(pfl->un.aryc[ijdis] & 00377»;

}

else {

ijdish = ijdis

-

1 ;

fracinth = fdish

*

«pfl->un.aryc[ijdisJ & 00377)

-

(pf1->un.aryc[ijdish] & 00377»;

}

pvalue = (pfl->un.aryc[ijdis] &00377)

+ fracintv + fracinth;

/*

pvalue should never leave [0,255], but 1n case ••• *j

if (pvalue > 255

I I

pvalue < 0) {

fprintf(stderr,"pvalue (%3d) out of range!O,pvalue);

pvalue = (pvalue<O ? 0 : 255);

}

j* predict the intensity

*/

if (dfdsum >= fdsum)

pf3~>un.aryc[ij] = pf1->un.aryc[ij] &00377; else

pf3->un.aryc[ij] = pvalue;

j* end prediction *j

/*

correct the predicted value by sending only change 1n

correction value

*/

(13)

-Aug 15 15:28 1985 gmcs25i.c Page 10

445 (pf3->un.aryc[ij] &00377»;

446

447 /* let's see the information bit rates if we were to send the

448 correction value instead of the change in correction value */

449 level

=

quantize(correction);

450 valhis[level]++;

451

452 dlevel

=

quantize(correction-prevdif);/* dlevel would be sent */

453 dvalhis[dlevel]++;

454 if (quanval[dlevel] == 0)

455 runlength++;

456 else {

457 addhis[runlength]++;

458 runlength = 0;

459 addsum++;

460 }

461 prevdif += quanval[dlevel];

462 value = (pf3->un.aryc[ij] &00377) + prevdif;

463

464 /* clip the pixel value */

465 if (value>255) value=2SS;

466 if (value<O) value=O;

467 pf4->un.aryc[ij]

=

value;

468

469 /* now that we have predicted the intensity value, let's update

470 the displacement estimate if the dfd is too big */

471 dfd[j] = (pf4->unoaryc[ij]

&

00377) - pvalue;

472 /* make *sum over all previous &adjacent pels */

473 dfdsum += (j==O) ? abs(dfd(j]) : abs(dfd(j]+dfd(j-l]);

474 fdsum += abs«pfl->un.aryc[ij] &00377)

475 - (p£4->un.aryc[ij]

&

00377»;

476 if (j) fdsum += abs«pf1->un.aryc[ij-l] &00377)

477 - (pf4->un.aryc[ij-l] & 00377»;

478 if (abs(dfd[j]) < motthres) {

479 mvva[j] = mvv;

480 mvha[j]

=

mvh;

481 iteration[O]++;

482 itertot[O]++;

483 }

484 else {

485 iter

=

0;

486 updatev

=

updateh

=

0.0001;

487 while «(updatev!=O.O)1 l(updateh!=O.O» && (iter<itermax») {

488 /* note using real multiplication, not product of signs */

489 dfd[j] = (pf4->un.aryc[ij]

&

00377) - pvalue;

490 if «ijdis<ncols)

I I

(ijdis+ncols>npels»

491 ldif

=

(ijdis<ncols)

492 ? «pfl->unoaryc[ijdis] &00377)

493 -(pf1->un.aryc[ijdis+ncols] &00377»

494 «pfl->unoaryc[ijdis-ncols] &00377)

495 -(pfl->un.aryc[ijdis] &00377»;

496 else

497 ldif «pfl->un.aryc[ijdis-ncols] &00377)

498 -(pf1->un.aryc[ijdis+ncols] &00377»/2;

499 if «ijdis%ncols==O)

I I

(ijdis%ncols==ncols-l»

(14)

501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556

? «pfl->un.aryc[ijdis+l] &00377)

-(pfl->un.aryc[ijdis] &00377»

«pf1->un.aryc[ijdis] &00377)

-(pf1->un.aryc[ijdis-l] & 00377»;

else

edif

=

«pfl->un.aryc[ijdis+1] &00377)

-(pfl->un.aryc[ijdis-l] &00377»/2;

dfd[j] = (abs(dfd[j]»signum) ? dfd[j] : 0;

ldif = (fabs(ldif»signum) ? Idif 0.0;

edif = (fabs(edif»signum) ? edif : 0.0;

gradnum++;

xgradsum += (edif*edif);

ygradsum += (ldif*ldif);

zgradsum += (edif*ldif);

if (fabs(edif) > fabs(xgradmax»

xgradmax = edif;

if (fabs(ldif) > fabs(ygradmax»

ygradmax = Idif;

if (fabs(edif*ldif) > fabs(zgradmax»

zgradmax = edi£*ldif;

updatev

=

epsilon*dfd[j]*ldif;

updateh

=

epsilon*dfd[j]*edif;

/* clip the updates to within cepsilon of zero */

if (updatev > cepsilon) updatev = cepsilon;

if (updatev < ncepsilon) updatev

=

ncepsilon;

if (updateh > cepsilon) updateh = cepsilon;

if (updateh < ncepsilon) updateh = ncepsilon;

mvv

-=

updatev;

mvh

-=

updateh;

imvv = «mvv>O) ?(int)(mvv+O.5):(int)(mvv-O.5»;

imvh = «mvh>O) ?(int)(mvh+0.5):(int)(mvh-0.5»;

/* let's interpolate the intensity values */

fdisv = imvv - mvv;

fdish

=

imvh - mvh;

/* if the displacement is outside the frame area,

clip to one row/column from edge and set the

fractional values to zero. We zero both fractional

values if we clip, because of indexing scheme. */

if (i+imvv < 0) {

imvv

=

-i;

fdisv

=

0.0;

fdish

=

0.0;

}

if (i+imvv > nrows-l) {

imvv = nrows-i-l;

fdisv = 0.0;

fdish = 0.0;

}

if (imvh > j) {

imvh = j;

fdisv = 0.0;

fdish

=

0.0;

}

if (imvh < j-ncols+l) {

imvh = j-ncols+l;

(15)

Aug 15 15:28 1985 gmcs2Sioc Page 12

557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612

fdish = 0.0;

}

ijdis = (i+imvv)*ncols + j-imvh;

if (imvv > mvv) {

ijdisv = ijdis - ncols;

fracintv = fdisv * «pfl->un.aryc[ijdisv]

&

00377)

- (pfl->un.aryc[ijdis]

&

00377»;

} else {

ijdisv = ijdis + ncols;

fracintv = fdisv

*

«pfl->un.aryc[ijdis]

&

00377)

- (pfl->un.aryc[ijdisv]

&

00377»;

}

if (imvh > mvh) {

ijdish = ijdis + 1;

fracinth = fdish

*

«pf1->un.aryc(ijdish]

&

00377)

- (pfl->un.aryc[ijdis] & 00377»;

} else {

ijdish = ijdis - 1;

fracinth = fdish

*

«pfl->unoaryc[ijdis]

&

00377)

- (pfl->un.aryc[ijdish]

&

00377»;

}

pvalue = (pfl->un.aryc[ijdis]

&

00377)

iter++; }

mvva[j] = mvv;

mvha[j] = mvh;

dfd[j] = (pf4->un.aryc[ij]

&

00377) - pvalue;

iteration[iter]++; itertot[iter]++;

if «updatev!=O.O)1 l(updatehl=O.O» noconv++;

if (fabs(mvva[j]) > fabs(maxmvv» {

maxmvv = mvva[j];

ijmaxv = ij;

}

if (fabs(mvha[j]) > fabs(maxmvh» {

maxmvh = mvha[j];

ijmaxh = ij;

}

} /*

end else

*/

} /*

end for j

*/

if (runlength != 0) {

addhis[runlength]++;

runlength = 0;

addsum++; }

prevdif = 0;

} /*

end for i

*/

if (frameout[O]

==

'y') {

k

=

pixout{pf3,fname3,"silent");

if (k<O) err =7;

k = pixout{pf4,fname4,"silent");

(16)

}

if (err>O) (

printf("err

=

%ld; error occurred In getting a file.O,err);

eXlt(-l);

j* print out maximum displacement values */

printf("Ohe maximum vertical displacement was: %6.1fO,maxmvv); printf("It occured at i= %3d, j= %3dO,ijmaxv/ncols,ijmaxv%ncols); printf("Ohe maximum horizontal displacement was: %6.1fO,maxmvh); printf("It occured at i= %3d, j= %3dO,ijmaxh/ncols,ijmaxh%ncols); printf("O);

for (i=O; i<=itermax; i++)

printf("The number of times we iterated %d times: %dO,i,iteration[i]);

printf("The number of times we did not converge was: %dO,noconv);

printf("Ohe number of times we clipped to stay in frame: %dO,nclips);

printf("The number of times mv* was reset in this frame: %dO,resetd);

the quantization levels of the diff values:O);

rep value bits # of occurrencesO);

i++) printf("O histogram of

printf(" upper limit

qbitsum = 0; pixsum = 0.0; entropy = 0.0; predpe12 = 0;

for (i=O; i<=quanlevs; }

ebitsum = (int)(pixsum*entropy + 0.5);

infobits2 += ebitsum; j*

printf("Ohe number of symbols sent is %6.0f.0,pixsum); printf("The quantizer bits sent would be %d.O,qbitsum); printf("The entropy would be %1.3fO,entropy);

printf("Thus the entropy bits sent would be %d.O,ebitsurn); */

print£("Ohe number of predictable pels was: i.dO,predpell); j*

printf("O histogram of the quantization levels:O);

printf(" upper limit rep value bits # of occurrencesO);

*/

qbitsum = 0;

pixsum = 0.0;

entropy = 0.0;

predpell = 0;

for (i=O; i<=quanlevs; i++)

pixsum += (quanval[i] ? valhis[i] : 0);

for (i=O; i<=quanlevs; i++) {

if «valhis[i] != 0)

&&

(quanval[i] != 0»

entropy += (valhis[i]/pixsum)*(-1.4427*log(valhis[i]/pixsum»;

if (quanval[i] == 0)

predpell += valhis[i];

qbitsum += (valhis[i]

*

quanlen[i]);

/-":

printf("%10d%14d%11d%16dO,quanept[i],quanval[i],quanlen[i],valhis[i]); */

(17)

Aug 1S 15:28 1985 gmcs25i.c Page 14

printf(ttO histogram of the skip codes (runlength addresses):O); entropy = OClO;

for (i=O; i<=ncols; i++)

if ( addhi s [i] != 0) {

entropy += (addhis[i]/addsum)*(-1.4427*log(addhis[i]/addsum); if (hardcopy[O] == 'v')

printf("length= %3d, number of occurrences= %6dO,i,addhis[i]); }

printf(tlOhe total number of runlengths sent is %6.0f.O,addsum); printf(ttThe entropy = %1.3f;",entropy);

ebitsum = (int)(addsum*entropy + 0.5);

bpf += ebitsum; addrbits ,+= ebitsum;

printf(ftthe entropy bits

=

%dO,ebitsum);

printf("the total bits in frame

=

%dO,bpf);

pixsum += (quanval[i] ? dvalhis[i] : 0);

for (i=O; i<=quanlevs; i++) {

if «dvalhis[i] != 0)

&&

(quanval[i] != 0»

entropy += (dvalhis[i]/pixsum)*(-1.4427*log(dvalhis[i]/pixsum»;

predpe12 += dvalhis[i];

qbitsum += (dvalhis[i] * quanlen[i]);

printf(U%9d%14d%11d%16dO,quanept[i],quanval[i],quanlen[i],dvalhis[i]); }

ebitsurn = (int)(pixsum*entropy + O.S);

bpf

=

ebitsum;

infobits += ebitsum;

printf("Ohe number of symbols sent is %6.0f.O,pixsum); printf("The quantizer bits sent would be %d.O,qbitsum); printf(ttThe entropy would be %1.3fO,entropy);

printf(ttThus the entropy bits sent would be %doO,ebitsum); printf(ttThe number of predictable pels was: %dO,predpe12);

%6.2fO,meansqer); %6.2f dBO,s_n); %6.2fO,meansqer); %6.2f dBO,s_n); }

meansqer = (double)sqerr/npels;

s n

=

10.0 * loglO(2SS*255/meansqer);

printf("Oean square error between previous frame: printf("Signal-to-noise when using previous frame:

1*

compute SNR's

*/

sqerr = 0;

for (i=O; i<nrows; i++)

for (j=O; j<ncols; j++) {

diff = igetpix(pf2,i,j) - igetpix(pfl,i,j); sqerr += diff*diff;

}

s n = 10.0

*

loglO(25S*25S/meansqer);

p~intf(ttmean square error between predicted frame:

printf(tlSignal-to-noise when using predicted frame: sqerr = 0;

diff

=

igetpix(pf2,i,j) - igetpix(pf3,i,j);

sqerr += diff*diff;

(18)

printf("O);

printf("total number times gradient computed in this frame: %8dO,gradnum);

printf("average X1!:X gradient: io7.1fO,xgradsum/gradnum);

print£("average y*y gradient: i.7.1fO,ygradsum/gradnum);

print£("average x*y gradient: i.7.1fO,zgradsum/gradnum);

printf("maximum x gradient: %6.1fO,xgradmax);

printf("maximum y gradient: %6.1fO,ygradmax);

printf("maximum x*y gradient: %6.1fO,zgradmax);

725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747

sqerr

=

0;

diff

=

igetpix(pf2,i,j) - igetpix(pf4,i,j);

sqerr += diff*diff;

}

s n

=

10.0

*

loglO(25S*255/meansqer);

-. f(U

prInt mean square error between corrected frame:

printf(USignal-to-noise when using corrected frame:

fflush(stdout);

(19)

Aug 15 15:28 1985 gmcs25i.c Page 16

748 /* do rest of frames */

749 ii=1;

750 while (ii<num) {

751 sprintf(fname2,"%s%d.hd",inpre,insuf+ii);

752 printf(" 14 %s: O,fname2);

753 sprintf(fname3,"%s%d.hd",outprel,insuf+ii);

754 sprint f (fname4,"%s%dGhd",out pre2 ,in suf +ii ) ;

755 for (i=O; i<npels; i++)

756 pf1->un.aryc(i] = pf4->un.aryc[i];

757 matfree(pf2);

758 k = pixin(pf2,fname2,"silent");

759 if (k<O) {

760 fprintf(stderr,"err =

%Id;

error 1n getting a file.O,k);

761 exit(-l);

762 }

763

764 mvv = 0.0;

765 mvh = 0.0;

766 maxmvv

=

0.0;

767 maxmvh = 000;

768 ijmaxv

=

0;

769 ijmaxh = 0;

770 dfd[O] = 0;

771 prevdif = 0;

772 runlength = 0;

773 addsum = OGO;

774 for (i=O; i<QLEV; i++) {

775 valhis[i] = 0;

776 dvalhis[i]

=

0;

777 }

778 for (i=O~ i<=NCOL; i++)

779 addhis[i] = 0;

780 for (i=O; i<MAXI; i++)

781 iteration[i] = 0;

782 noconv = 0;

783 nclips = 0;

784 resetd = 0;

785 gradnum = 0;

786 xgradsum

=

0.0;

787 ygradsum = 0.0;

788 zgradsum = 0.0;

789 xgradmax = 0.0;

790 ygradmax = 0.0;

791 zgradmax = 0.0;

792

793 /* reduce interframe noise

*/

794 if (terminal(O] == 'v')

795 fprintf(stderr9 " s t a r t noise suppressionO);

796 for (i=O; i<ncols*nrows; i++) {

797 peldif = (pf2->un.aryc[i]

&

00377)

798 - (pfl->un.aryc[i]

&

00377);

799 dframe[i] = (*threshold)(peldif);

800

801

/*

now do intraframe noise suppression (isolated ptG rejection)

(20)

}

} else

pf2->un.aryc[ij] = (pf1->un.aryc[ij]

&

}

else

&

1J = i*ncols+ncols-l;

if(!dframe[ij-l] && !dframe[ij-ncols] && !dframe[ij+neols]) {

if «dframe[ij]>S) && (terminal[O] == 'v'»

fprintf(stderr,"ij=%6d,dif=%4dO,ij,dframe[ij]);

pf2->un.aryc[ij] = (pfl->un.aryc[ij] &00377);

}

else

pf2->un.aryc[ij] = (pf1->un.aryc[ij] &00377) + dframe[ij];

for (j=l; j<ncols-l; j++) { ij = i*ncols+j;

if«dframe[ij-l]==O)&&(dframe[ij-ncols]==O)&&

(dframe[ij+l]==O)&&(dframe[ij+ncols]==O» {

if «dframe[ij]>S) && (terminal[O] == 'v'»

pf2->un.aryc[ij] = (pfl->un.aryc[ij] & 00377);

}

else

&

}

/*

do last row similar to first row

*/

i

=

nrows-l;

ij = i*ncols;

&

for (j=l; j<ncols-l; j++) { ij = i*ncols+j;

if(!dframe(ij-1]

&&

tdframe(ij-ncols]

&&

fdframe[ij+l]) {

if «dframe[ij]>S)

&&

(terminal[O] == 'v'»

p£2->un.aryc(ij] = (pfl->un.aryc[ij]

&

00377);

/*

first row a special case; skip first and last pel

*/

pf2->un.aryc[0] = (pf1->un.aryc[0]

&

00377) +

dframe[O]-for (j=l; j<ncols-1; j++) ,

if(!dframe[j-1] && !dframe[j+1] && !dframe[j+ncols]) {

if «dframe[j]>5) && (terminal[O] ==

'v'»

fprintf(stderr,"ij=%6d,dif=%4dO,j,dframe[j]);

pf2->un.aryc[j] = (pfl->un.aryc[j] &00377);

}

else

pf2->un.aryc[j]

=

(pf1->un.aryc[j] &00377) + dframe[j];

j = neols-I;

pf2->un.arye[j] = (pf1->un.arye[j]

&

00377) + dframe[j];

/*

continue through next-to-last row

*/

for (i=1; i<nrows-l; i++) { ij = i*ncols;

if(Idframe[ij-ncolsJ && Idframe[ij+1] && !dframe[ij+neols]) {

if «dframe[ij]>5)

&&

(terminal[O] == 'v'»

&

00377);

(21)

Aug 15 15:28 1985 gmcs25i.c Page 18

859 ij

=

nrows*ncols-l;

860 pf2->un.aryc[ij]

=

(pfl->unGaryc[ij]

&

(22)

j* None *j

mvv = omvva[j+l];

else

/* predict and transmit values on this line */

for(j=O; j<ncols; j++) {

ij :: i*ncols + j;

dfdsum :: abs(odfd[j])+abs(odfd[j+l])+abs(odfd[j+2]);

fdsum :: abs(fdif[j])+abs(fdif[j+l])+abs(fdif[j+2]);

if (j==O)

mvv = omvva[j]; /* NW one *j

if (j==O)

mvh = omvha[j+l]; j* None */

else

mvh = omvha]j] ; /* NW one ...':/

imvv :: «mvv>O) ? (int)(mvv+O.S) (int)(mvv-O.S»;

imvh :: «mvh>O) ? (int)(mvh+O.S) : (int)(mvh-O.5»;

/*

let's interpolate the intensity values */

fdisv

=

imvv - mvv;

fdish :: imvh - mvh;

/*

if the displacement is outside the frame area,

values if we clip, because of indexing scheme. */

if (i+imvv < 0) {

imvv = -i;

fdisv :: 0.0;

j* recurse and predict *j

if (terminat[O] == 'v')

fprintf(stderr,"start recursing and predictingO); for(i=O; i<nrows; i++) {

j* load the DFDs and FDs from previous line */

if (i > 0)

forIj=O; j<ncols; j++) {

odfd[j+l] = dfd[j];

fdif[j+1] = (pf4->un.aryc[(i-1)*ncols+j] &00377)

- (pfl->un.aryc[(i-1)*ncols+j]

&

00377);

omvva[j+l] = mvva[j]; omvha[j+1] = mvha[j];

}

odfd[O]

=

(odfd[I]+odfd[2]»>1;

odfd[ncols+l] = (od£d[ncols-l]+odfd[ncols]»>l; fdif[O] :: (fdi£[1]+fdif[2]»>1;

fdif[ncols+l] :: (fdif[ncols-l]+fdif[ncols]»>1; omvva[O] = omvva[l];

omvva[ncols+l]

=

omvva[ncols];

omvha[O] = omvha[l];

omvha[ncols+1J = omvha[ncols]; }

else /* once we start looking at sequences, this might disappear? */

fore j=O; j<ncols ; j++) {

odfd[j+1] :: 512;

fdif[j+l]

=

0;

omvva[j+l] = 0;

omvha[j+l] = 0;

(23)

}

else {

fracintv = fdisv ~': _{«pfl->un.aryc[ijdis]} _& ₀₀₃₇₇₎

-

(pfl->un.aryc[ijdisv] & 00377»;

}

if (imvh > mvh) {

ijdish = ijdis + 1 ;

fracinth

=

fdish

_*

«pfl->un.aryc[ijdish] & 00377)

-

(pfl->un.aryc[ijdis] & 00377»;

} else {

ijdish = ijdis

-

1 ;

fracinth

=

fdish * «pfl->un.aryc[ijdis] & 00377)

-

(pfl->un.aryc[ijdish] & 00377»;

Aug 15 15:28 1985 gmcs25iGc Page 20

917 fdish = 0.0;

918 }

919 if (i+imvv > nrows-l)

920 imvv = nrows-i-l;

921 fdisv

=

0.0;

922 fdish

=

0.0;

923 }

924 if (imvh > j) {

925 imvh

=

j;

926 fdisv = 000;

927 fdish = 0.0;

928 }

929 if (imvh < j-ncols+l) (

930 imvh = j-ncols+l;

931 fdisv = 0.0;

932 fdish = 0.0;

933 }

934 ijdis = (i+imvv)*ncols + j-imvh;

935 if (imvv > mvv) {

936 ijdisv = ijdis - ncols;

937 fracintv = fdisv

*

«pfl->un.aryc[ijdisv]

&

00377)

938 - (pfl->un.aryc[ijdis]

&

00377»;

939 940 941 942 943 944 945 946 947 948 949 950 951 952 953

954 }

955 pvalue = (pfl->un.aryc[ijdis]

&

00377)

956 + fracintv + fracinth;

957

/*

pvalue should never leave [0,255], but 10 case •••

*/

958 if (pvalue > 255

I I

pvalue < 0) (

959 fprintf(stderr,"pvalue (%3d) out of range!O,pvalue);

960 pvalue = (pvalue<O ?

a :

255);

961 }

962

963

/*

predict the intensity

*/

964 if (dfdsum >= fdsum)

965 pf3->un.aryc[ij] = pfl->uoGaryc[ij]

&

00377;

966 else

967 pf3->un.aryc[ij] pvalue;

968

/*

end prediction

*/

969

970

/*

correct the predicted value by sending only change 1n

971 correction value

*/

(24)

-973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028

(pf3->un.aryc[ij] &00377»;

/* let's see the information bit rates if we were to send the

correction value instead of the change 1n correction value ~I

level = quantize(correction); valhis[level]++;

dlevel = quantize(correction-prevdif);/* dlevel would be sent ~/

dvalhis[dlevel]++;

if (quanval[dlevel] == 0)

runlength++; else {

addhis[runlength]++; runlength = 0;

addsum++; }

prevdif += quanval[dlevel];

value = (pf3->un.aryc[ij]

&

00377) + prevdif;

/* clip the pixel value */

if (value>2SS) value=255;

if (value<O) value=O;

pf4->un.aryc[ij] = value;

/* now that we have predicted the intensity value, let's update

the displacement estimate if the dfd is too big */

dfd[j] = (pf4->un.aryc[ij] &00377) - pvalue;

/* make *surn over all previous & adjacent pels

*/

dfdsum += (j==O) ? abs(dfd(j]) : abs(dfd[j]+dfd[j-l]);

fdsum += abs«pfl->un.aryc[ij] & 00377)

- (pf4->un.aryc[ij] &00377»;

if (j) fdsum += abs«pfl->un.aryc[ij-l] &00377)

- (pf4->un.aryc[ij-l]

&

00377»;

if (abs(dfd[j]) < motthres) {

mvva[j] = mvv;

mvha[j] = mvh; iteration[O]++; itertot[O]++; }

else {

iter = 0;

updatev = updateh = 0.0001;

r.-1hile «(updatev!=O.O)II(updatehl=O.O» && (iter<itermax» {

/* note using real multiplication, not product of signs */

dfd[j] = (pf4->un.aryc[ij]

&

00377) - pvalue;

if «ijdis<ncols)

I

I (ijdis+ncols>npels»

ldif = (ijdis<ncols)

? «pfl->un.aryc[ijdis]

&

00377)

-(pfl->un.aryc[ijdis+ncols] &00377»)

«pfl->un.aryc[ijdis-ncols]

&

00377)

-(pfl->un.aryc[ijdis]

&

00377»;

else

ldif = «pfl->un.aryc[ijdis-ncols] &00377)

-(pfl->un.aryc[ijdis+ncols]

&

00377»/2;

if «ijdis%ncols==O) I

I

(ijdis%ncols==ncols-l»

(25)

Aug 15 15:28 1985

1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 lOSS 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084

gmcs25icc Page 22

? «pfl->un.aryc[ijdis+l]

&

00377)

-(pfl->un.aryc[ijdis]

&

00377»

«pfl->un.aryc[ijdis]

&

00377)

-(pfl->un.aryc[ijdis-l]

&

00377»;

else

edif = «pfl->un.aryc[ijdis+l]

&

00377)

-(pfl->unoaryc[ijdis-1]

&

00377»/2;

dfd[j] = (abs(dfd[j]»signum) ? dfd[j] : 0;

Idif = (fabs(ldif»signum) ? Idif 000;

edif

=

(fabs(edif»signum) ? edif : 000;

gradnum++;

xgradsum += (edif*edif);

ygradsum += (ldif*ldif);

zgradsum += (edif*ldif);

if (fabs(edif) > fabs(xgradmax»

xgradmax

=

edif;

if (fabs(ldif) > fabs(ygradmax»

ygradmax

=

Idif;

if (fabs(edif*ldif) > fabs(zgradmax»

zgradmax = edif*ldif;

updatev

=

epsilon*dfd[j]*ldif;

updateh

=

epsilon*dfd[j]*edif;

/*

clip the updates to within cepsilon of zero

*/

if (updatev > cepsilon) updatev

=

cepsiloo;

if (updatev < ncepsilon) updatev = ncepsilon;

if (updateh > cepsilon) updateh

=

cepsilon;

if (updateh < ncepsilon) updateh

=

ncepsilon;

mvv

-=

updatev;

mvh

-=

updateh;

imvv = «mvv>O) ?(int)(mvv+OoS):(int)(mvv-OoS»;

imvh

=

«mvh>O) ?(int)(mvh+O.S):(int)(mvh-O.5»;

/*

let's interpolate the intensity values

*/

fdisv

=

imvv - mvv;

fdish = imvh - mvh;

/*

if the displacement is outside the frame area,

values if we clip, because of indexing scheme~

*/

if (i+imvv < 0) {

lmvv = -i;

fdisv = 0.0;

fdish = 0.0;

}

if (i+imvv > nrows-l)

imvv = nrows-i-!;

fdisv = 000;

fdish

=

000;

}

if (imvh > j) {

imvh = j;

fdisv = 0.0;

fdish = 0.0;

}

if (imvh < j-ncols+l) {

imvh = j-ncols+l;

(26)

1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140

fdish

=

0.0;

}

ijdis = (i+imvv)*ncols + j-imvh;

if (imvv > mvv) {

ijdisv = ijdis - ncols;

fracintv

=

fdisv * «pf1->un.aryc[ijdisv] &00377)

- (pf1->un.aryc[ijdis] &00377»;

} else {

fracintv

=

fdisv

*

- (pfl->un.aryc[ijdisv] &00377»;

}

if (imvh > mvh) {

ijdish = ijdis + 1;

fracinth = fdish

*

«pf1->un.aryc[ijdish] &00377)

- (pf1->un.aryc[ijdis] &00377»;

} else {

ijdish = ijdis - 1;

fracinth = fdish

*

- (pf1->un.aryc[ijdish] &00377»;

}

pvalue = (pfl->un.aryc[ijdis] &00377)

iter++; }

mvva[j] = mvv;

mvha[j]

=

mvh;

dfd[j] = (pf4->un.aryc[ij] &00377) - pvalue;

iteration[iter]++; itertot[iter]++;

if «updatev!=O.O)1

I

(updateh!=O.O» noconv++;

if (fabs(mvva[j]) > fabs(maxmvv» {

maxmvv = mvva[j];

ijmaxv

=

ij;

}

if (fabs(mvha[j]) > fabs(maxmvh» {

maxmvh

=

mvha[j];

ijmaxh = ij;

}

} /*

end else

*/

} /*

end for j

*/

if (runlength != 0) {

addhis[runlength]++;

runlength = 0;

addsum++; }

prevdif

=

0;

} /*

end for i

*/

if (frameout[O]

==

'y') {

" ( f 3""1 It)_

k

=

p1xout pf3, name , 51 ent ,

if (k<O) err =13;

" ( f 4""1

")-k

=

p1xout pf4, name , 51 ent ,

(27)

Aug 15 15:28 1985 gmcs25i.c Page 24

1141

1142 if (err>O) {

1143 printf(tt e r r = %ld; error occurred 1n getting a file.O,err);

1144 exit(-l);

1145 1146

1147 /* print out maximum displacement values

*/

1148 printf("Ohe maximum vertical displacement was: %6.1fO,maxmvv);

1149 printf("It occured at i= %3d, j= %3dO,ijmaxv/ncols,ijmaxv%nco1s);

1150 printf("Ohe maximum horizontal displacement was: %6.1fO,maxmvh);

1151 printf("It occured at i= %3d, j= %3dO,ijmaxh/ncols,ijmaxh%ncols);

1152

1153 printf("O);

1154 for (i=O; i<=itermax; i++)

1155 printf("The number of times we iterated %d times: %dO,i,iteration(i]);

1156 printf("The number of times we did not converge was: %dO,noconv);

1157 printf("Ohe number of times we clipped to stay in frame: %dO,nclips);

1158 printf("The number of times mv* was reset in this frame: %dO,resetd);

1159

1160 /*

1161 print£("O histogram of the quantization levels:O);

1162 printf(" upper limit rep value bits iF of occurrencesO);

1163 ...':/

1164 qbitsum = 0;

1165 pixsum = 000;

1166 entropy =

OeD;

1167 predpel1 = 0;

1168 for (i=O; i<=quanlevs; i++)

1169 pixsum += (quanval(i] ? valhis(i] : 0);

1170 for (i=O; i<=quanlevs; i++) {

1171 if «valhis[i] != 0)

&&

(quanval[i] != 0»

1172 entropy += (valhis[i]/pixsum)*(-lo4427*log(valhis[i]/pixsum»;

1173 if (quanval[i] == 0)

1174 predpel1 += valhis(i];

1175 qbitsum += (valhis[i]

*

quanlen[i]);

1176 /*

1177 printf("%10d%14d%11d%16dO,quanept[i],quanval[i],quanlen[i],valhis(i]);

1178 */

1179 }

1180 ebitsum = (int)(pixsum*entropy + Oes);

1181 infobits2 += ebitsum;

1182 /*

1183 printf("Ohe number of symbols sent is %6.0f.O,pixsum);

1184 printf("The quantizer bits sent would be %d.O,qbitsum);

1185 printf("The entropy would be %1.3fO,entropy);

1186 printf("Thus the entropy bits sent would be %d.O,ebitsum);

1187 */

1188 printf(UOhe number of predictable pels was: %dO,predpell);

1189

1190 printf("O histogram of the quantization levels of the diff values:O);

1191 printf(" upper limit rep value bits iF of occurrencesO);

1192 qbitsum = 0;

1193 pixsum = 0.0;

1194 entropy = 0.0;

1195 predpel2

=

0;

(28)

%6.2£O,meansqer); %6.2£ dBO,s_n); %6.2fO,meansqer); %6.2f dBO,s_n);

pixsum += (quanval[i] ? dvalhis[i] : 0);

(i=O; i<=quanlevs; i++) {

if «dvalhis[i] != 0)

&&

(quanval[i] !=

0»

entropy += (dvalhis[i]/pixsum)~~(-1.4427*log(dvalhis[i]/pixsum»;

predpe12 += dvalhis[i];

qbitsum += (dvalhis[i] -,'~ quanlenji j Lt

printf(tt%9d%14d%11d%16dO,quanept[i],quanval[i],quanlen[i],dvalhis[i]);

printf("Q histogram of the skip codes (runlength addresses):O);

entropy = 0.0;

for (i=O; i<=ncols; i++)

if (addbi sli ] != 0) {

entropy += (addhis[i]/addsum)*(-1.4427*log(addhis[i]/addsum»;

if (hardcopy[O]

==

'v')

printf("length= %3d, number of occurrences= %6dO,i,addhis[i]); }

ebitsum = (int)(pixsum*entropy + O.S);

bpf = ebitsum;

infobits += ebitsum;

printf(ttOhe number of symbols sent is %6.0f.O,pixsum); printf("The quantizer bits sent would be %d.O,qbitsum); printf("The entropy would be %1.3fO,entropy);

printf("Thus the entropy bits sent would be %d.O,ebitsum); printf("The number of predictable pels was: %dO,predpelZ);

}

printf("Ohe total number of runlengths sent is %6.0f.O,addsum);

printf("The entropy

=

%1.3f;",entropy);

ebitsum = (int)(addsum*entropy + 0.5);

bpf += ebitsum;

addrbits += ebitsum;

printf("the entropy bits

=

%dO,ebitsum);

printf("the total bits in frame = %dO,bpf);

/* compute SNR's */

sqerr = 0;

diff = igetpix(pf2,i,j) - igetpix(pfl,i,j); sqerr += dif£*diff;

for

}

s n = 10.0

*

loglO(25S*2SS/meansqer);

printf("Oean square error between previous frame: printf("Signal-to-noise when using previous frame:

}

s n = 10.0 * logI0(25S*255/meansqer);

printf("mean square error betwe~n predi

_7ted

frame:

printf("Signal-to-noise when uSing predicted frame: sqerr = 0;

diff = igetpix(pf2,i,j) - igetpix(pf3,i,j);

sqerr += diff*diff; 1197

(29)

Aug 1S 15:28 1985 gmcs25i.c Page 26

fflush(stdout);

printf("Ohe total number of st. diff bits (so far): %9dO,infobits2);

printf("The total number of dif-of-dif bits (so far): %9dO,infobits);

printf("The total number of address bits (so far): %9dO,addrbits);

ii++;

} /* end while */

printf("O);

for (i=O; i<=itermax; i++)

printf(ltThe number of times we iterated %d times: %dO,i,itertot[i]);

} 1*

end main */ printf(ltO);

printf("total number times gradient computed in this frame: %8dO,gradnum);

printf("average x*x gradient: %7.1fO,xgradsum/gradnum);

printf("average y*y gradient: %7.1fO,ygradsum/gradnum);

printf("average x*y gradient: %7.1fO,zgradsum/gradnum);

printf("maximum x gradient: %601fO,xgradmax);

printf("maximum y gradient: %601fO,ygradmax);

printf(ttmaximum x*y gradient: %6.1fO,zgradmax);

%6.2fO,meansqer); %6.2f dBO,s_n);

sqerr = 0;

diff = igetpix(pf2,i,j) - igetpix(pf4,i,j);

sqerr += diff*diff;

}

meansqer

=

(double)sqerr/npels;

s n

=

10.0

*

log10(25S*25S/meansqer);

p~intf("mean square error between corrected frame:

printf("Signal-to-noise when using corrected frame: 1253

(30)

setpar()

/*

This procedure sets quantizer parameters. The number of quantizer

levels lS assumed to be strictly less than QLEV. */

printf("Ohe files opened by setpar were:O);

/*

Set Parameters For Quantizer

*/

/* a file descriptor and a function to

open each file

*/

/* indexing variables */

int i;

FILE *fr,*openfile();

fr = openfile("Oile containing quantizing information: 0);

fscanf(fr,"%dO,&quanlevs);

for(i=O; i<=quanlevs; i++)

fscanf(fr,"%d%d%dO,&quanept[i],&quanval[i],&quanlen[i]); fclose(fr);

(31)

Aug 15 15:28 1985 gmcs2SiQc Page 28

1312

1313

/*

Quantizer Section

*/

1314 1315

1316 quancize(del)

1317 int del;

1318 1319

1320 inc

k;

1321

1322 for (k=«quanlevs»I)+l); k<=quanlevs; k++)

1323 if (abs(del) < quanept[k])

1324 return(del>O ? k : quanlevs-k);

1325 return(del>O ? quanlevs :

a );

(32)

1328

1329

/*

Openfiles

*/

1330

1331

/*

This procedure opens files from which parameters can be read

*/

1332

1333 FILE *openfile(string)

1334 char *string;

1335

1336 {

1337 char filename[40];

1338 FILE *fopen(),*fptr;

1339

1340 fprintf(stderr,string);

1341 scanf("%s",filename);

1342 if«fptr = fopen(filename,"rtt

_»

==

NULL) {

1343 fprintf(stderr,"can't open %sO,filename);

1344 exit(1);

1345 }

1346 printf(" %sO,filename);

1347 return(fptr);

1348 }

(33)

Aug 15 15:28 1985 gmcs2Si.c Page 30

} }

/*

Threshold Selection Section

*/

fprintf(stderr,"Ohich threshold function do you want?O);

fprintf(stderr," enter

a

for thresholdp()O);

fprintf(stderr," enter 1 for threshold1()O);

fprintf(stderr," enter 4 for thresholdr()O);

fprintf(stderr," enter 5 for thresholdO()O);

scanf("%d",&thrnum);

printf("Ohe threshold function selected was IF%dO,thrnum);

switch (thrnum) { case 0:

*threshold = thresholdp;

break; case 1:

*threshold = thresholdl;

break; case 2:

*threshold = threshold2;

break; case 3:

*threshold

=

threshold3;

break; case 4:

*threshold = thresholdr;

break;

case 5:

*threshold

=

thresholdO;

break; default:

fprintf(stderr,"Non-existent threshold function selected!O); exit(-l);

/*

used to select threshold function

*/

/*

threshold functions

*/

/*

threshold function

*/

thrnwn,

thresholdp(), thresholdl(), threshold2(), threshold3(), thresholdr(), thresholdO(); int

selthr(threshold) int (**threshold)(); 1350

(34)

/*

The interframe differences are clipped via threshold(). The data is

histogrammed. The values to be sent are quantized. The entropy of

the pel difference values is calculated. The addressing requirements

are calculated. The maximum motion vector components and their locations

are determined.

*/

/*

This.simulation is derived from an algorithm developed by Netravali and

Robbl~s.at Bel~ Labs. Thre~ papers should be noted: "Motion-Compensated

Te17vlsiO~ Codlng: Part I, BSTJ, March 1979, pp. 631-670; "Interframe

Coding USlng Movement Compensation," ICC, June 1979, pp. 23.4.1-23.4.5; and

"Spatial Subsampling in Motion-Compensated Television Coders," BSTJ, Oct.

1982, pp. 1897-1898. We are seeking the algorithm which will give the

minimum transmitted bit rate.

*/

/* This simulation version has the following unique:

1) the motion vector is maintained as a real number.

2) the displaced intensity prediction *is* interpolated.

3) the previous motion vector is in the direction of motion.

4) the motion vector is updated using real multiplication.

S) pvalue is calculated using interpolation.

6) intraframe noise suppression (isolated point rejection).

7) mv update is product.

8) dfd[j] is updated after mv is updated.

9) mv* is updated dependent on dfd[j] only.

10) dfd[j] is interpolated in iterative mv-update loop.

11) predictor switches between CR and MC dependent on dfdsum/fdsum.

12) only two mv* correction choices (OK or iterate). */

#include

f~include

#include

f~define

fidefine

f~define f~define

/*

#" of columns in frame */

/*

# of rows in frame

*/

/*

maximum quantization levels

*/

/* maximum number of iterations */

/*

number of quantization levels (must be odd)~

Includes positive and negative levels

*/

/*

value at end of each quantization range */

/*

quantization value (level) for each

quanti-zation range

*/

/*

length of code word at quantization level */

/* difference frame for isolated pt. reject */

/*

how many predictions for mv

*/

/*

how far vertically the pel moved

*/

/*

how far horizontally the pel moved */

<stdio.h> <math.h>

"/usr/local/dsp/dsp2.h"

NCOL 448

NROW 282

QLEV 40

MAXI 40

quanlevs, quanept[QLEV] , quanval[QLEV] , quanlen[QLEV] , dframe[NCOL*NROW], mvnum[2][NCOL*NROW]; mvver[2][NCOL*NROW], mvhor[2][NCOL*NROW]; int

float 1

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

20

21 22 23 24 25 26 27 28 29 30 31 32 33 34 3S 36 37 38 39 40 41 42 43 44

4S

(35)

Aug 15 15:20 1985 gmcs27i.c Page 2

49

50 maine)

51 {

52 char fnamel[40],

1*

the previously transmitted frame */

S3 fname2[40],

1*

the next frame to be transmitted */

&

uncorrected output frame */

S5 fname4[40],

1*

the predicted

&

corrected output frame */

56 inpre[30], /* the prefix common to every input frame */

S7 outprel[31], /* the prefix common to every fname3 */

58 outpre2[31]; /* the prefix common to every fname4 */

59 struct data fl,~~pfl; /* the numbers correspond to numbers above */

63 register int i,j; /* indexing variables */

64 int k,

65 err,

66 nrows, /* the number of rows 1n the source frame */

67 ncols~ /* the number of cols 1n the source frame */

68 npels~ /* the number of pels 10 the source frame */

69 peldif, /* interframe difference used in noise red */

70 (*threshold)()~ /* pointer to a threshold function */

71 correction,

1*

difference between correct value and

72 predicted value */

73 prevdif, /* previous difference of differences */

74 dlevel,

1*

quantization level of dif of prevdifs */

75 level, /* quantization level of correction value */

76 runlength,

1*

number of skipped zero values */

77 value, /* the corrected value in the predicted frame

*/

78 pvalue, /* displaced predicted value */

79 imvv, /* vertical component of the motion vector */

80 imvh, /* horizontal component of the motion vector */

81 nimvv~ /* prediction for verte comp of mv next frame

*1

82 nimvh, /* prediction for horzo comp of mv next frame

*/

83 nclips, /* number of times nimvv/nimvh was clipped */

84 resetd, /* number of times mv* is reset each frame */

85 mvnumO,

1*

number of my's never predicted */

86 mvnum1, /* number of my's once predicted */

87 mvnum2, /* number of mv's twice predicted */

88 mvnum3, /* number of mv's predicted more than twice */

89 fracintv, /* vertical interpolation fraction */

90 fracinth,

1*

horizontal interpolation fraction */

91 1J, /* present pel address */

92 ijmaxv, /* location of maximum vertical displacement */

93 ijmaxh, /* location of maximum horizontal displace */

94 ijdis, /* displaced pel address in previous frame */

95 nijdis, /* predicted location of this intensity in next

96 frame */

97 ijdisv, /* nearest vertical location to ijdis */

98 ijdish, /* nearest horizontal location to ijdis */

99 signum, I"": threshold on "sign-of-value" for update */

100 motthres, /* motion threshold */

101 fdsum, /* sum of frame differences */

102 dfdsum, /* sum of displaced frame differences */

(36)

maxmvv, maxmvh, addsum, pixsum, entropy, meansqer, s n; hardcopy[lO], frameout[10], terminal[lO], 104 lOS 106 107 108 109 110 III 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 float char odfd[NCOL+2], /"': fdif[NCOL+2], /* num, /* insuf, /* 11, /* jj,jl, /* valhis[QLEV], /* dvalhis[QLEV], /* addhis[NCOL+1], /* qbitsum, /* ebitsum, /* infobits, /* infobits2, /* addrbits, /* bpf, /* predpe11, /* predpe12, /* itermax,

/*

iter,

/*

iteration[MAXI],/* itertot[MAXI], /* gradnum, /* noconv,

/*

diff, /* sqerr; /* updatev, /* updateh, /* edif,

/*

ldif, /* xgradsum, /* ygradsum, /* zgradsum, /* xgradmax, /* ygradmax,

/*

zgradmax, /* epsilon,

/*

cepsilon, /* ncepsilon, /* fdisv, /* fdish,

/*

mvv, /* mvh,

/*

mvva[NCOL+2],

/*

mvha[NCOL+2], /* omvva[NCOL+2], /* omvha[NCOL+2], /*

/*

/ ..'(

/*

/i~ /~(

/*

the DFDs in previous line */

the FDs in previous line

*/

number of frames to be processed */

the integer part of every image

*/

frame indexing variable */

mv interframe array index variables */

used to histogram values transmitted

*/

used to histogram dif values transmitted

*/

used to histogram runlengths transmitted */

total number of quantizer bits transmitted */

total number of entropy bits in a frame */

total number of dif-of-dif bits */

total number of st. diff bits (approx.) */

total number of address bits */

total bits per frame (info + addr) */

number of predictable pels in each frame */

the maximum number of iterations «MAXI) */

which iteration is being performed */

how many times we've iterated this far */

total number of iterations in sequence */

total number of gradients in a frame */

incremented each time we don't converge */

frame difference (error) */

sum of squared errors in a frame */

the vertical update component. */

the horizontal update component */

element difference */

line difference */

total square of edge gradients per frame */

total square of line gradients per frame */

total square of cross gradients per frame */

maximum edge gradient in a frame */

maximum line gradient in a frame */

maximum cross gradient in a frame */

converge coefficient */

maximum update value */

additive inverse of maximum update value */

fractional part of vertical displacement */

fractional part of horizontal displacement

*/

vertical component of the motion vector */

horizontal component of the motion vector */

vertical mv's on present line */

horizontal mv's on present line */

vertical mv's on previous line */

horizontal mv's on previous line */

maximum vertical displacement */

maximum horizontal displacement

*/

number of runlengths sent */

total number of pels scanned */

p*log{p) of quantization words

*/

mean square error in a frame */

signal to noise ratio in a frame

*/

"verbose" ('v' ••• ) or "short" */ "yes" ('y' ••• ) or "no" "k/

(37)

Aug 15 15:20 1985 gmcs27i.c Page 4

160 161

(38)

162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 181 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216

/

,'._" _{set file pointers} "it: /

pf1 = &£1;

pf2 = &f2;

pf3 = &£3;

pf4 = &£4;

I"~·,.. _{get file} _names )':/

err = 0;

k = getline(fnamel,"name of previous frame:O)·

printf("Ohe name of the previous frame: ' a/Of105 , name1)·,

if (k<O) err =1;

k = getline(inpre,"prefix common to every input frame:O);

print f C'The prefix common to every input fr ame:rame: a/105O·,lnpre ;)

if (k<O) err =9;

k = getline(outpre1,"prefix common to every outputl frame:O);

if (k<O) err =10;

k

=

getline(outpre2,"prefix common to every output2 frame:O);

if (k<O) err =11;

fprintf(stderr,"suffix of first frame to be transmitted (integer):O); scanf("%d",&insuf);

printf("The suffix of the first frame to transmit: %2dO,insuf);

fprintf(stderr,"the number of frames to process (integer):O); scanf("%d",&num);

printf("The number of frames to process was: %2dO,num);

sprintf(fname2,"%s%d.hd",inpre,insuf); sprintf(fname3,"%s%d.hd",outpre1,insuf); sprintf(fname4,"%s%d.hd",outpre2,insuf);

fprintf(stderr,"What is the convergence coefficient?O); scanf("%f",&epsilon);

printf("The convergence coefficient was: %.6fO,epsilon);

fprintf(stderr,"What is the maximum update value?O);

scanf("%f",&cepsilon);

printf("The maximum update value was: %.6fO,cepsilon);

ncepsilon = -cepsilon;

fprintf(stderr,"What is the update-displacement threshold?O); scanf("%d",&motthres);

printf("The update-displacement threshold was: %ldO,motthres);

fprintf(stderr,"What is the signum threshold?O); scanf("%d",&signum);

printf("The signum threshold was: %ldO,signum);

fprintf(stderr,"What is the maximum number of iterations?O); scanf("%d",&itermax);

printf("The maximum number of iterations was: %2dO,itermax);

fprintf(stderr,"Oo you want verbose or short hardcopy output?O)j scanf("%s",hardcopy);

printf("The output was made: %sO,hardcopy);

fprintf(stderr,"do you want the output frames saved?O)j scanf("%s",frameout);

printf("Were the output frames saved? %sO,frameout)j