A Path Towards Quantum Advantage in Training Deep Generative Models with Quantum Annealing

(1)

Copyright D-Wave Systems Inc

A Path Towards Quantum Advantage

in Training Deep Generative Models

with

Quantum Annealing

(2)

How to use quantum annealers to provide quantum

advantage on real applications?

• The good

:

1. Quantum annealers are competitive with state-of-the-art classical

solvers on natively defined problems

• The bad

:

2. Relatively small number of available qubits

3. Quasi two-dimensional (spatially local) connectivities

4. Control and thermal errors

• The ugly:

1. We still don’t know how to exploit the good to solve useful problems

[Mandra, Katzgraber, QST 3; King et al, arXiv:1701.04579;

Hen et al. PRA 92; Albash and Lidar, PRX 8]

[Troyer, Katzgraber: AQC2019 talks;

Albash et al. QST 4]

(3)

1. Optimization:

-

Quantum annealing developed as a quantum heuristic for

optimization

-

Well-established suite of tools for benchmarking:

time-to-solution(target) measures, planted solutions…

-

Divide-and-conquer and embedding algorithms face large overheads

2. Sampling:

-

A more recent application for QA, potential not fully understood

-

Benchmarking is more subtle: KL-divergences are expensive to

compute, approximate sampling sufficient for practical applications…

-

Focus on machine learning applications and techniques for

‘embeddings’

(4)

• Quantum annealers simulate

a transverse field Ising model

immersed in a thermal bath

• The required technology is being developed: advanced annealing

schedules (pauses, fast quenches, reverse anneals) (

D-Wave/QEO

)

-

The use of D-Wave quantum annealers as quantum Boltzmann

samplers recently

demonstrated in material simulations

[Amin, PRA 92; Marshall et

al., PR Applied 11]

thermalization

freeze-out

frozen

[Harris et al., Science 361; King et al., Nature 560]

Quantum annealers are special purpose devices, exploit

what they do best:

(5)

• Boltzmann Machines (BM) approximate data

distributions as

thermal states of classical spin-systems

-

Boltzmann Machines (BM) and QA: a

perfect match

?

• Training BM requires

Boltzmann sampling

:

-

State-of-the-art sampling techniques:

(Persisent) Contrasting Divergence

(P)CD

-

Difficult to scale to large, more powerful BM

p

✓

(

z

)

⌘

e

E

✓

(

z

)

/Z

✓

,

Z

✓

⌘

X

z

e

E

✓(

z

)

<latexit sha1_base64="hEX79KaerF0YK8s7Ujvrgi5Dj7c=">AAAERHicbVNLbxMxEHayPEp4pXDkYhFVKlIJu6gS10ooUo5FatqKOkT2rpNYXT9qe8s21v42bvwBTvwHbogDF4Sz2aTZpCOtd/x9M99YnjFRKTM2DH82msG9+w8e7jxqPX7y9Nnz9u6LUyMzHdNBLFOpzwk2NGWCDiyzKT1XmmJOUnpGLj/O+bNrqg2T4sTeKDrkeCLYmMXYemjU5mqEiJ1Si/fR7A1E9Cpj19Cbo1/c2946WRTvPi/36OAAoqsMJ3AFLXMdMhkfoRm8Q2DU7oTdsDS47USV0wGVHY92m99QIuOMU2HjFBtzEYXKDh3WlsUpLVooM1Th+BJP6IV3BebUDF15LwXc80gCx1L7T1hYousZbqKxmrI4r+lU2TXIZGTMJpn2FWslhdQcp5QPXebXeg7R+JLaOoa54dhOi9beOqqSsfJ/U4/1bSO+S3xLwdxwUgfnmlbKdEOB8PlxBf0aS86xSBwSovCLFBknVEOEijrd8+xcixDXKza4PuYVG+PU9Tfp2RqJyMy3fB6R0DFSnlXzhuG0gAsor8qMXb6Mmq2g2RIqVbwY9+preouBcgui9IsVRZVhqRQVqafylvKbu2Df/wr23gpWt0deAFb7WjS37kQvY3r52mWNHMohMoxD/6DKwASXB/MjH20O+LZz+r4bhd3o02Hn6LAa/h3wCrwG+yACH8AR6INjMAAx+AH+NpqNIPge/Ap+B38Woc1GlfMS1Cz49x9aeHwY</latexit><latexit sha1_base64="hEX79KaerF0YK8s7Ujvrgi5Dj7c=">AAAERHicbVNLbxMxEHayPEp4pXDkYhFVKlIJu6gS10ooUo5FatqKOkT2rpNYXT9qe8s21v42bvwBTvwHbogDF4Sz2aTZpCOtd/x9M99YnjFRKTM2DH82msG9+w8e7jxqPX7y9Nnz9u6LUyMzHdNBLFOpzwk2NGWCDiyzKT1XmmJOUnpGLj/O+bNrqg2T4sTeKDrkeCLYmMXYemjU5mqEiJ1Si/fR7A1E9Cpj19Cbo1/c2946WRTvPi/36OAAoqsMJ3AFLXMdMhkfoRm8Q2DU7oTdsDS47USV0wGVHY92m99QIuOMU2HjFBtzEYXKDh3WlsUpLVooM1Th+BJP6IV3BebUDF15LwXc80gCx1L7T1hYousZbqKxmrI4r+lU2TXIZGTMJpn2FWslhdQcp5QPXebXeg7R+JLaOoa54dhOi9beOqqSsfJ/U4/1bSO+S3xLwdxwUgfnmlbKdEOB8PlxBf0aS86xSBwSovCLFBknVEOEijrd8+xcixDXKza4PuYVG+PU9Tfp2RqJyMy3fB6R0DFSnlXzhuG0gAsor8qMXb6Mmq2g2RIqVbwY9+preouBcgui9IsVRZVhqRQVqafylvKbu2Df/wr23gpWt0deAFb7WjS37kQvY3r52mWNHMohMoxD/6DKwASXB/MjH20O+LZz+r4bhd3o02Hn6LAa/h3wCrwG+yACH8AR6INjMAAx+AH+NpqNIPge/Ap+B38Woc1GlfMS1Cz49x9aeHwY</latexit><latexit sha1_base64="hEX79KaerF0YK8s7Ujvrgi5Dj7c=">AAAERHicbVNLbxMxEHayPEp4pXDkYhFVKlIJu6gS10ooUo5FatqKOkT2rpNYXT9qe8s21v42bvwBTvwHbogDF4Sz2aTZpCOtd/x9M99YnjFRKTM2DH82msG9+w8e7jxqPX7y9Nnz9u6LUyMzHdNBLFOpzwk2NGWCDiyzKT1XmmJOUnpGLj/O+bNrqg2T4sTeKDrkeCLYmMXYemjU5mqEiJ1Si/fR7A1E9Cpj19Cbo1/c2946WRTvPi/36OAAoqsMJ3AFLXMdMhkfoRm8Q2DU7oTdsDS47USV0wGVHY92m99QIuOMU2HjFBtzEYXKDh3WlsUpLVooM1Th+BJP6IV3BebUDF15LwXc80gCx1L7T1hYousZbqKxmrI4r+lU2TXIZGTMJpn2FWslhdQcp5QPXebXeg7R+JLaOoa54dhOi9beOqqSsfJ/U4/1bSO+S3xLwdxwUgfnmlbKdEOB8PlxBf0aS86xSBwSovCLFBknVEOEijrd8+xcixDXKza4PuYVG+PU9Tfp2RqJyMy3fB6R0DFSnlXzhuG0gAsor8qMXb6Mmq2g2RIqVbwY9+preouBcgui9IsVRZVhqRQVqafylvKbu2Df/wr23gpWt0deAFb7WjS37kQvY3r52mWNHMohMoxD/6DKwASXB/MjH20O+LZz+r4bhd3o02Hn6LAa/h3wCrwG+yACH8AR6INjMAAx+AH+NpqNIPge/Ap+B38Woc1GlfMS1Cz49x9aeHwY</latexit><latexit sha1_base64="hEX79KaerF0YK8s7Ujvrgi5Dj7c=">AAAERHicbVNLbxMxEHayPEp4pXDkYhFVKlIJu6gS10ooUo5FatqKOkT2rpNYXT9qe8s21v42bvwBTvwHbogDF4Sz2aTZpCOtd/x9M99YnjFRKTM2DH82msG9+w8e7jxqPX7y9Nnz9u6LUyMzHdNBLFOpzwk2NGWCDiyzKT1XmmJOUnpGLj/O+bNrqg2T4sTeKDrkeCLYmMXYemjU5mqEiJ1Si/fR7A1E9Cpj19Cbo1/c2946WRTvPi/36OAAoqsMJ3AFLXMdMhkfoRm8Q2DU7oTdsDS47USV0wGVHY92m99QIuOMU2HjFBtzEYXKDh3WlsUpLVooM1Th+BJP6IV3BebUDF15LwXc80gCx1L7T1hYousZbqKxmrI4r+lU2TXIZGTMJpn2FWslhdQcp5QPXebXeg7R+JLaOoa54dhOi9beOqqSsfJ/U4/1bSO+S3xLwdxwUgfnmlbKdEOB8PlxBf0aS86xSBwSovCLFBknVEOEijrd8+xcixDXKza4PuYVG+PU9Tfp2RqJyMy3fB6R0DFSnlXzhuG0gAsor8qMXb6Mmq2g2RIqVbwY9+preouBcgui9IsVRZVhqRQVqafylvKbu2Df/wr23gpWt0deAFb7WjS37kQvY3r52mWNHMohMoxD/6DKwASXB/MjH20O+LZz+r4bhd3o02Hn6LAa/h3wCrwG+yACH8AR6INjMAAx+AH+NpqNIPge/Ap+B38Woc1GlfMS1Cz49x9aeHwY</latexit>

E

_✓

(

z

) =

X

l

z

_l

h

_l

+

X

l<m

W

_lm

z

_l

z

_m

,

h

,

W

₂

_{

✓

_}

<latexit sha1_base64="5P3i0k7gukOIWYicaoJcv7SjhIU=">AAAEPnicbVNdaxQxFE27ftT1q9VHX4KlULGUXSn4olCQhX2s0O0WmmVIZjK7Q/NlktHthPlt+gf8Af4B38QHX3xQ8O7MtN3ZGkhy5px7T0LuHWZE5nyv921tvXPr9p27G/e69x88fPR4c+vJidO5jfko1kLbU0YdF5niI595wU+N5VQywcfs/N1CH3/k1mVaHfsLwyeSTlWWZjH1QEWb0y4eRIT5Gfd0lxQv8FuMictlJHABcwbzZU0E8QbLEuMxINiLKkLuYfIhpwkOhKV4Vu7VYFxikilMQuNMymhzu7ffqwa+CfoN2EbNOIq21j+TRMe55MrHgjp31u8ZPwnU+iwWvOyS3HFD43M65WcAFZXcTUL1IiXeASbBqbYwlccVu5wRppaaWRbPWz5NdotyOUuzaW7hxNaRSltJBZeTkMPazmGWnnPf5qh0kvpZ2d1ZZk2SGthdOxYKxqA+8oaDu5CsTS48vdZixYHJxXUV/xRrKalKAlGqhEWrXDJuMSFlWx6AuvBiLAzKFW1IZaPGVIThqlwsiYQVUO5FRMJTYkA1i4JRUeKamjfHpGF+GVVcUcUlVbmAmQT3Jb+6mUItVLi8krhxmdCqEe1MX0vw8T8a6t/QgK5oc33lmvAWzuJzH47tZcxgvvRYUSBz+EMyiU1UBya0uhi0fH+1wW+Ck1f7/d5+//3B9uFB0/wb6Bl6jnZRH71Gh2iIjtAIxegr+oX+oL+dL53vnR+dn3Xo+lqT8xS1Ruf3P7gweZ0=</latexit><latexit sha1_base64="5P3i0k7gukOIWYicaoJcv7SjhIU=">AAAEPnicbVNdaxQxFE27ftT1q9VHX4KlULGUXSn4olCQhX2s0O0WmmVIZjK7Q/NlktHthPlt+gf8Af4B38QHX3xQ8O7MtN3ZGkhy5px7T0LuHWZE5nyv921tvXPr9p27G/e69x88fPR4c+vJidO5jfko1kLbU0YdF5niI595wU+N5VQywcfs/N1CH3/k1mVaHfsLwyeSTlWWZjH1QEWb0y4eRIT5Gfd0lxQv8FuMictlJHABcwbzZU0E8QbLEuMxINiLKkLuYfIhpwkOhKV4Vu7VYFxikilMQuNMymhzu7ffqwa+CfoN2EbNOIq21j+TRMe55MrHgjp31u8ZPwnU+iwWvOyS3HFD43M65WcAFZXcTUL1IiXeASbBqbYwlccVu5wRppaaWRbPWz5NdotyOUuzaW7hxNaRSltJBZeTkMPazmGWnnPf5qh0kvpZ2d1ZZk2SGthdOxYKxqA+8oaDu5CsTS48vdZixYHJxXUV/xRrKalKAlGqhEWrXDJuMSFlWx6AuvBiLAzKFW1IZaPGVIThqlwsiYQVUO5FRMJTYkA1i4JRUeKamjfHpGF+GVVcUcUlVbmAmQT3Jb+6mUItVLi8krhxmdCqEe1MX0vw8T8a6t/QgK5oc33lmvAWzuJzH47tZcxgvvRYUSBz+EMyiU1UBya0uhi0fH+1wW+Ck1f7/d5+//3B9uFB0/wb6Bl6jnZRH71Gh2iIjtAIxegr+oX+oL+dL53vnR+dn3Xo+lqT8xS1Ruf3P7gweZ0=</latexit><latexit sha1_base64="5P3i0k7gukOIWYicaoJcv7SjhIU=">AAAEPnicbVNdaxQxFE27ftT1q9VHX4KlULGUXSn4olCQhX2s0O0WmmVIZjK7Q/NlktHthPlt+gf8Af4B38QHX3xQ8O7MtN3ZGkhy5px7T0LuHWZE5nyv921tvXPr9p27G/e69x88fPR4c+vJidO5jfko1kLbU0YdF5niI595wU+N5VQywcfs/N1CH3/k1mVaHfsLwyeSTlWWZjH1QEWb0y4eRIT5Gfd0lxQv8FuMictlJHABcwbzZU0E8QbLEuMxINiLKkLuYfIhpwkOhKV4Vu7VYFxikilMQuNMymhzu7ffqwa+CfoN2EbNOIq21j+TRMe55MrHgjp31u8ZPwnU+iwWvOyS3HFD43M65WcAFZXcTUL1IiXeASbBqbYwlccVu5wRppaaWRbPWz5NdotyOUuzaW7hxNaRSltJBZeTkMPazmGWnnPf5qh0kvpZ2d1ZZk2SGthdOxYKxqA+8oaDu5CsTS48vdZixYHJxXUV/xRrKalKAlGqhEWrXDJuMSFlWx6AuvBiLAzKFW1IZaPGVIThqlwsiYQVUO5FRMJTYkA1i4JRUeKamjfHpGF+GVVcUcUlVbmAmQT3Jb+6mUItVLi8krhxmdCqEe1MX0vw8T8a6t/QgK5oc33lmvAWzuJzH47tZcxgvvRYUSBz+EMyiU1UBya0uhi0fH+1wW+Ck1f7/d5+//3B9uFB0/wb6Bl6jnZRH71Gh2iIjtAIxegr+oX+oL+dL53vnR+dn3Xo+lqT8xS1Ruf3P7gweZ0=</latexit><latexit sha1_base64="5P3i0k7gukOIWYicaoJcv7SjhIU=">AAAEPnicbVNdaxQxFE27ftT1q9VHX4KlULGUXSn4olCQhX2s0O0WmmVIZjK7Q/NlktHthPlt+gf8Af4B38QHX3xQ8O7MtN3ZGkhy5px7T0LuHWZE5nyv921tvXPr9p27G/e69x88fPR4c+vJidO5jfko1kLbU0YdF5niI595wU+N5VQywcfs/N1CH3/k1mVaHfsLwyeSTlWWZjH1QEWb0y4eRIT5Gfd0lxQv8FuMictlJHABcwbzZU0E8QbLEuMxINiLKkLuYfIhpwkOhKV4Vu7VYFxikilMQuNMymhzu7ffqwa+CfoN2EbNOIq21j+TRMe55MrHgjp31u8ZPwnU+iwWvOyS3HFD43M65WcAFZXcTUL1IiXeASbBqbYwlccVu5wRppaaWRbPWz5NdotyOUuzaW7hxNaRSltJBZeTkMPazmGWnnPf5qh0kvpZ2d1ZZk2SGthdOxYKxqA+8oaDu5CsTS48vdZixYHJxXUV/xRrKalKAlGqhEWrXDJuMSFlWx6AuvBiLAzKFW1IZaPGVIThqlwsiYQVUO5FRMJTYkA1i4JRUeKamjfHpGF+GVVcUcUlVbmAmQT3Jb+6mUItVLi8krhxmdCqEe1MX0vw8T8a6t/QgK5oc33lmvAWzuJzH47tZcxgvvRYUSBz+EMyiU1UBya0uhi0fH+1wW+Ck1f7/d5+//3B9uFB0/wb6Bl6jnZRH71Gh2iIjtAIxegr+oX+oL+dL53vnR+dn3Xo+lqT8xS1Ruf3P7gweZ0=</latexit>

x

<latexit sha1_base64="Ych406Ue8ZWFYscOoav0Y6eUyOc=">AAAB6HicbVDLTgJBEOzFF+IL9ehlIjHxRHaRRI8kXjxCIo8ENmR26IWR2dnNzKyREL7AiweN8eonefNvHGAPClbSSaWqO91dQSK4Nq777eQ2Nre2d/K7hb39g8Oj4vFJS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj27nffkSleSzvzSRBP6JDyUPOqLFS46lfLLlldwGyTryMlCBDvV/86g1ilkYoDRNU667nJsafUmU4Ezgr9FKNCWVjOsSupZJGqP3p4tAZubDKgISxsiUNWai/J6Y00noSBbYzomakV725+J/XTU1440+5TFKDki0XhakgJibzr8mAK2RGTCyhTHF7K2EjqigzNpuCDcFbfXmdtCpl76pcaVRLtWoWRx7O4BwuwYNrqMEd1KEJDBCe4RXenAfnxXl3PpatOSebOYU/cD5/AONRjPI=</latexit>

visible

units

z

<latexit sha1_base64="mQiNxs+0BhHXM7p62on2cheOo+8=">AAAB6HicbVDLTgJBEOzFF+IL9ehlIjHxRHaRRI8kXjxCIo8ENmR26IWR2dnNzKwJEr7AiweN8eonefNvHGAPClbSSaWqO91dQSK4Nq777eQ2Nre2d/K7hb39g8Oj4vFJS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj27nffkSleSzvzSRBP6JDyUPOqLFS46lfLLlldwGyTryMlCBDvV/86g1ilkYoDRNU667nJsafUmU4Ezgr9FKNCWVjOsSupZJGqP3p4tAZubDKgISxsiUNWai/J6Y00noSBbYzomakV725+J/XTU1440+5TFKDki0XhakgJibzr8mAK2RGTCyhTHF7K2EjqigzNpuCDcFbfXmdtCpl76pcaVRLtWoWRx7O4BwuwYNrqMEd1KEJDBCe4RXenAfnxXl3PpatOSebOYU/cD5/AOZZjPQ=</latexit>

latent

units

=

<latexit sha1_base64="71OX+Oy9Jd3D1BrB44x6lUL2szs=">AAAB6XicbVDLSgNBEOyNrxhfUY9eBoPgKezGgF6EgBePUcwDkiXMTnqTIbOzy8ysGEL+wIsHRbz6R978GyfJHjSxoKGo6qa7K0gE18Z1v53c2vrG5lZ+u7Czu7d/UDw8auo4VQwbLBaxagdUo+ASG4Ybge1EIY0Cga1gdDPzW4+oNI/lgxkn6Ed0IHnIGTVWur9+6hVLbtmdg6wSLyMlyFDvFb+6/ZilEUrDBNW647mJ8SdUGc4ETgvdVGNC2YgOsGOppBFqfzK/dErOrNInYaxsSUPm6u+JCY20HkeB7YyoGeplbyb+53VSE175Ey6T1KBki0VhKoiJyext0ucKmRFjSyhT3N5K2JAqyowNp2BD8JZfXiXNStm7KFfuqqVaNYsjDydwCufgwSXU4Bbq0AAGITzDK7w5I+fFeXc+Fq05J5s5hj9wPn8AZPWNOQ==</latexit>

x

Employ quantum annealers for faster, more scalable sampling

[Smolensky, ’86]

(6)

• Connectivity is a very important factor

-

Generative performance of BM on

Chimera graph:

disappointing

• Rely on technological improvements

-

Chimera (D-Wave) -> Pegasus (D-Wave) -> QEO program

-

Only

quasi two-dimensional

connectivities available for the

foreseeable future

• Rely on common embedding techniques

-

Sampling quality decreases dramatically, likely not scalable

[Dumoulin et al.,

AAAI Conf, ’14]

[MNIST 50k

handwritten

digits]

(7)

=

x

data in

latent features out

• Quantum/classical

joint training

-

Extract the most suitable features for the quantum device

-

Hard-coded specification of the connectivity not required

[Vinci et al.,

arXiv:1912.02119]

=

x

p

_✓

(

x

)

<latexit sha1_base64="4DoC2WG1SBx7hhIegpsKY0uD+qw=">AAAD6HicbZLdbtMwFMe9ho9Rvlq45CaimjRuqgRN4nYSqtTLIa3bpKWqbMdpTf0l22FZrbwDd4gLbriAx+A5eBucJt2aFktJTn7/c/7Hsg9SjBobRX8POsGDh48eHz7pPn32/MXLXv/VhZG5xmSCJZP6CkFDGBVkYqll5EppAjli5BItP1b65ReiDZXi3N4qMuVwLmhGMbQezXp9NXMJsgtiYXmcFO/CcNYbRMNovcL9IG6CAWjW2azf+ZOkEuecCIsZNOY6jpSdOqgtxYyU3SQ3REG8hHNy7UMBOTFTt957GR55koaZ1P4RNlzT7Qo311AtKC5aPk11C5kcZXSea9+x1VJIzSEjfOpy/27XIA2XxLYZ5IZDuyi7R9tUpZnyX9PO9UeL/EnyPQdzy1EbVp5WSrbjgHi1XUFusOQcitQlQpT+JUXOEdFhkpRteeTVygshNyp3tDHkjYohc+NdebUlJmhV3bnPSEmWKK+q6sIgK8MaFU2bzBWbrNUdWm3Q2sWbce++5VdPlKuFerruJKIMZVI0ol7Ie8n//A/7+2+wj+6wut9yDaz2vUhh3bne5IyKrcPyk16EiaE8VLM6MYXrjfmRj3cHfD+4eD+Mo2H86WRwetIM/yF4A96CYxCDD+AUjMEZmAAMbsAP8Av8Dj4HX4Nvwfc6tXPQ1LwGrRX8/Afr8lmA</latexit><latexit sha1_base64="4DoC2WG1SBx7hhIegpsKY0uD+qw=">AAAD6HicbZLdbtMwFMe9ho9Rvlq45CaimjRuqgRN4nYSqtTLIa3bpKWqbMdpTf0l22FZrbwDd4gLbriAx+A5eBucJt2aFktJTn7/c/7Hsg9SjBobRX8POsGDh48eHz7pPn32/MXLXv/VhZG5xmSCJZP6CkFDGBVkYqll5EppAjli5BItP1b65ReiDZXi3N4qMuVwLmhGMbQezXp9NXMJsgtiYXmcFO/CcNYbRMNovcL9IG6CAWjW2azf+ZOkEuecCIsZNOY6jpSdOqgtxYyU3SQ3REG8hHNy7UMBOTFTt957GR55koaZ1P4RNlzT7Qo311AtKC5aPk11C5kcZXSea9+x1VJIzSEjfOpy/27XIA2XxLYZ5IZDuyi7R9tUpZnyX9PO9UeL/EnyPQdzy1EbVp5WSrbjgHi1XUFusOQcitQlQpT+JUXOEdFhkpRteeTVygshNyp3tDHkjYohc+NdebUlJmhV3bnPSEmWKK+q6sIgK8MaFU2bzBWbrNUdWm3Q2sWbce++5VdPlKuFerruJKIMZVI0ol7Ie8n//A/7+2+wj+6wut9yDaz2vUhh3bne5IyKrcPyk16EiaE8VLM6MYXrjfmRj3cHfD+4eD+Mo2H86WRwetIM/yF4A96CYxCDD+AUjMEZmAAMbsAP8Av8Dj4HX4Nvwfc6tXPQ1LwGrRX8/Afr8lmA</latexit><latexit sha1_base64="4DoC2WG1SBx7hhIegpsKY0uD+qw=">AAAD6HicbZLdbtMwFMe9ho9Rvlq45CaimjRuqgRN4nYSqtTLIa3bpKWqbMdpTf0l22FZrbwDd4gLbriAx+A5eBucJt2aFktJTn7/c/7Hsg9SjBobRX8POsGDh48eHz7pPn32/MXLXv/VhZG5xmSCJZP6CkFDGBVkYqll5EppAjli5BItP1b65ReiDZXi3N4qMuVwLmhGMbQezXp9NXMJsgtiYXmcFO/CcNYbRMNovcL9IG6CAWjW2azf+ZOkEuecCIsZNOY6jpSdOqgtxYyU3SQ3REG8hHNy7UMBOTFTt957GR55koaZ1P4RNlzT7Qo311AtKC5aPk11C5kcZXSea9+x1VJIzSEjfOpy/27XIA2XxLYZ5IZDuyi7R9tUpZnyX9PO9UeL/EnyPQdzy1EbVp5WSrbjgHi1XUFusOQcitQlQpT+JUXOEdFhkpRteeTVygshNyp3tDHkjYohc+NdebUlJmhV3bnPSEmWKK+q6sIgK8MaFU2bzBWbrNUdWm3Q2sWbce++5VdPlKuFerruJKIMZVI0ol7Ie8n//A/7+2+wj+6wut9yDaz2vUhh3bne5IyKrcPyk16EiaE8VLM6MYXrjfmRj3cHfD+4eD+Mo2H86WRwetIM/yF4A96CYxCDD+AUjMEZmAAMbsAP8Av8Dj4HX4Nvwfc6tXPQ1LwGrRX8/Afr8lmA</latexit><latexit sha1_base64="4DoC2WG1SBx7hhIegpsKY0uD+qw=">AAAD6HicbZLdbtMwFMe9ho9Rvlq45CaimjRuqgRN4nYSqtTLIa3bpKWqbMdpTf0l22FZrbwDd4gLbriAx+A5eBucJt2aFktJTn7/c/7Hsg9SjBobRX8POsGDh48eHz7pPn32/MXLXv/VhZG5xmSCJZP6CkFDGBVkYqll5EppAjli5BItP1b65ReiDZXi3N4qMuVwLmhGMbQezXp9NXMJsgtiYXmcFO/CcNYbRMNovcL9IG6CAWjW2azf+ZOkEuecCIsZNOY6jpSdOqgtxYyU3SQ3REG8hHNy7UMBOTFTt957GR55koaZ1P4RNlzT7Qo311AtKC5aPk11C5kcZXSea9+x1VJIzSEjfOpy/27XIA2XxLYZ5IZDuyi7R9tUpZnyX9PO9UeL/EnyPQdzy1EbVp5WSrbjgHi1XUFusOQcitQlQpT+JUXOEdFhkpRteeTVygshNyp3tDHkjYohc+NdebUlJmhV3bnPSEmWKK+q6sIgK8MaFU2bzBWbrNUdWm3Q2sWbce++5VdPlKuFerruJKIMZVI0ol7Ie8n//A/7+2+wj+6wut9yDaz2vUhh3bne5IyKrcPyk16EiaE8VLM6MYXrjfmRj3cHfD+4eD+Mo2H86WRwetIM/yF4A96CYxCDD+AUjMEZmAAMbsAP8Av8Dj4HX4Nvwfc6tXPQ1LwGrRX8/Afr8lmA</latexit>

q

(

z

_|

x

)

<latexit sha1_base64="l8Z4DgwEPbx7zX95Cw0l40G3S7I=">AAAD6nicbZLLbtNAFIanMZcSbildsrGIKpVNZKNKbCuhSFkWqWkrdaJoxh4no8ytM2NwYvwS7BALNizgKXgO3oax46R2wkgeH3//Of8cjQ9WjBobBH8POt6Dh48eHz7pPn32/MXL3tGrKyNTHZFxJJnUNxgZwqggY0stIzdKE8QxI9d48aHUrz8RbagUl3apyISjmaAJjZB1aNo7vpvmEKs5LfxTuPK/+DB7O+31g0FQLX8/COugD+p1MT3q/IGxjFJOhI0YMuY2DJSd5EhbGjFSdGFqiELRAs3IrQsF4sRM8qr7wj9xJPYTqd0jrF/RZkU+08g1GGUtn7q6hUyKEzpLtTuxdaSQmiNG+CRP3d6uwRotiG0zxA1Hdl50T5pUxYlyb9POdZeL3V3yPQez5LgNS08rJdtxwLxsV5DPkeQciTiHQhRukyLlmGgfwqItD51aemGcD4sdbYR4rUaI5aNdedUQIV4Ri8qMmCRQOVWVPwyxwl+jrD4mybNN1mqLVhtUuTgz7twbftjOS74WqrjYSkQZyqSoRT2X95L7+B92/7/G5ahue75veQ2sdmeRzOaXepMzzBqX5WY986Gh3FfTdWKMqsbcyIe7A74fXL0bhMEg/HjWPz+rh/8QvAZvwCkIwXtwDkbgAoxBBJbgB/gFfnvM++p9876vUzsHdc0xaC3v5z9ktloy</latexit><latexit sha1_base64="l8Z4DgwEPbx7zX95Cw0l40G3S7I=">AAAD6nicbZLLbtNAFIanMZcSbildsrGIKpVNZKNKbCuhSFkWqWkrdaJoxh4no8ytM2NwYvwS7BALNizgKXgO3oax46R2wkgeH3//Of8cjQ9WjBobBH8POt6Dh48eHz7pPn32/MXL3tGrKyNTHZFxJJnUNxgZwqggY0stIzdKE8QxI9d48aHUrz8RbagUl3apyISjmaAJjZB1aNo7vpvmEKs5LfxTuPK/+DB7O+31g0FQLX8/COugD+p1MT3q/IGxjFJOhI0YMuY2DJSd5EhbGjFSdGFqiELRAs3IrQsF4sRM8qr7wj9xJPYTqd0jrF/RZkU+08g1GGUtn7q6hUyKEzpLtTuxdaSQmiNG+CRP3d6uwRotiG0zxA1Hdl50T5pUxYlyb9POdZeL3V3yPQez5LgNS08rJdtxwLxsV5DPkeQciTiHQhRukyLlmGgfwqItD51aemGcD4sdbYR4rUaI5aNdedUQIV4Ri8qMmCRQOVWVPwyxwl+jrD4mybNN1mqLVhtUuTgz7twbftjOS74WqrjYSkQZyqSoRT2X95L7+B92/7/G5ahue75veQ2sdmeRzOaXepMzzBqX5WY986Gh3FfTdWKMqsbcyIe7A74fXL0bhMEg/HjWPz+rh/8QvAZvwCkIwXtwDkbgAoxBBJbgB/gFfnvM++p9876vUzsHdc0xaC3v5z9ktloy</latexit><latexit sha1_base64="l8Z4DgwEPbx7zX95Cw0l40G3S7I=">AAAD6nicbZLLbtNAFIanMZcSbildsrGIKpVNZKNKbCuhSFkWqWkrdaJoxh4no8ytM2NwYvwS7BALNizgKXgO3oax46R2wkgeH3//Of8cjQ9WjBobBH8POt6Dh48eHz7pPn32/MXL3tGrKyNTHZFxJJnUNxgZwqggY0stIzdKE8QxI9d48aHUrz8RbagUl3apyISjmaAJjZB1aNo7vpvmEKs5LfxTuPK/+DB7O+31g0FQLX8/COugD+p1MT3q/IGxjFJOhI0YMuY2DJSd5EhbGjFSdGFqiELRAs3IrQsF4sRM8qr7wj9xJPYTqd0jrF/RZkU+08g1GGUtn7q6hUyKEzpLtTuxdaSQmiNG+CRP3d6uwRotiG0zxA1Hdl50T5pUxYlyb9POdZeL3V3yPQez5LgNS08rJdtxwLxsV5DPkeQciTiHQhRukyLlmGgfwqItD51aemGcD4sdbYR4rUaI5aNdedUQIV4Ri8qMmCRQOVWVPwyxwl+jrD4mybNN1mqLVhtUuTgz7twbftjOS74WqrjYSkQZyqSoRT2X95L7+B92/7/G5ahue75veQ2sdmeRzOaXepMzzBqX5WY986Gh3FfTdWKMqsbcyIe7A74fXL0bhMEg/HjWPz+rh/8QvAZvwCkIwXtwDkbgAoxBBJbgB/gFfnvM++p9876vUzsHdc0xaC3v5z9ktloy</latexit><latexit sha1_base64="l8Z4DgwEPbx7zX95Cw0l40G3S7I=">AAAD6nicbZLLbtNAFIanMZcSbildsrGIKpVNZKNKbCuhSFkWqWkrdaJoxh4no8ytM2NwYvwS7BALNizgKXgO3oax46R2wkgeH3//Of8cjQ9WjBobBH8POt6Dh48eHz7pPn32/MXL3tGrKyNTHZFxJJnUNxgZwqggY0stIzdKE8QxI9d48aHUrz8RbagUl3apyISjmaAJjZB1aNo7vpvmEKs5LfxTuPK/+DB7O+31g0FQLX8/COugD+p1MT3q/IGxjFJOhI0YMuY2DJSd5EhbGjFSdGFqiELRAs3IrQsF4sRM8qr7wj9xJPYTqd0jrF/RZkU+08g1GGUtn7q6hUyKEzpLtTuxdaSQmiNG+CRP3d6uwRotiG0zxA1Hdl50T5pUxYlyb9POdZeL3V3yPQez5LgNS08rJdtxwLxsV5DPkeQciTiHQhRukyLlmGgfwqItD51aemGcD4sdbYR4rUaI5aNdedUQIV4Ri8qMmCRQOVWVPwyxwl+jrD4mybNN1mqLVhtUuTgz7twbftjOS74WqrjYSkQZyqSoRT2X95L7+B92/7/G5ahue75veQ2sdmeRzOaXepMzzBqX5WY986Gh3FfTdWKMqsbcyIe7A74fXL0bhMEg/HjWPz+rh/8QvAZvwCkIwXtwDkbgAoxBBJbgB/gFfnvM++p9876vUzsHdc0xaC3v5z9ktloy</latexit>

p

✓

(

x

|

z

)

<latexit sha1_base64="9UMQ33OoCCIo5NFwcxexC9JqB6k=">AAAD6HicbZLLbtNAFIanMZcSbgks2VhElcomslGlbiuhSFkWqWkrdaJoZjxOTOammTF1bfwO7BALNizgMXgO3oax46Rxwki2j7//nP+MZg5WLDE2CP4edLwHDx89PnzSffrs+YuXvf6rSyNTTeiESCb1NUaGskTQiU0so9dKU8Qxo1d4+aHSrz5TbRIpLuydolOO5iKJE4KsQ7NeX80gtgtqkX8Msy8wfzfrDYJhUC9/PwibYACadT7rd/7ASJKUU2EJQ8bchIGy0wJpmxBGyy5MDVWILNGc3rhQIE7NtKj3XvpHjkR+LLV7hPVrul1RzDVSi4RkLZ+muoVMiuNknmrXsdVSSM0Ro3xapO7drsEaLaltM8QNR3ZRdo+2qYpi5b6mneuOFruT5HsO5o7jNqw8rZRsxwHzaruC3hLJORJRAYUo3UuKlGOqfQjLtjxyauWFcTEqd7Qx4o1KECvGu3K+JUKcu0uvMiIaQ+VUVV0YYqW/QlnTJi6ydVa+Qfka1S7OjDv3Lb/VSBUroY7LjUSVSZgUjagX8l5yP//D7v4b7KINVvdbXgGrXS+a2eJCr3NG2dZhzQqY+dAk3HdDXydGqN6YG/lwd8D3g8v3wzAYhh9PBmcnzfAfgjfgLTgGITgFZ2AMzsEEEHALfoBf4Lf3yfvqffO+r1I7B03Na9Ba3s9/sRFZug==</latexit><latexit sha1_base64="9UMQ33OoCCIo5NFwcxexC9JqB6k=">AAAD6HicbZLLbtNAFIanMZcSbgks2VhElcomslGlbiuhSFkWqWkrdaJoZjxOTOammTF1bfwO7BALNizgMXgO3oax46Rxwki2j7//nP+MZg5WLDE2CP4edLwHDx89PnzSffrs+YuXvf6rSyNTTeiESCb1NUaGskTQiU0so9dKU8Qxo1d4+aHSrz5TbRIpLuydolOO5iKJE4KsQ7NeX80gtgtqkX8Msy8wfzfrDYJhUC9/PwibYACadT7rd/7ASJKUU2EJQ8bchIGy0wJpmxBGyy5MDVWILNGc3rhQIE7NtKj3XvpHjkR+LLV7hPVrul1RzDVSi4RkLZ+muoVMiuNknmrXsdVSSM0Ro3xapO7drsEaLaltM8QNR3ZRdo+2qYpi5b6mneuOFruT5HsO5o7jNqw8rZRsxwHzaruC3hLJORJRAYUo3UuKlGOqfQjLtjxyauWFcTEqd7Qx4o1KECvGu3K+JUKcu0uvMiIaQ+VUVV0YYqW/QlnTJi6ydVa+Qfka1S7OjDv3Lb/VSBUroY7LjUSVSZgUjagX8l5yP//D7v4b7KINVvdbXgGrXS+a2eJCr3NG2dZhzQqY+dAk3HdDXydGqN6YG/lwd8D3g8v3wzAYhh9PBmcnzfAfgjfgLTgGITgFZ2AMzsEEEHALfoBf4Lf3yfvqffO+r1I7B03Na9Ba3s9/sRFZug==</latexit><latexit sha1_base64="9UMQ33OoCCIo5NFwcxexC9JqB6k=">AAAD6HicbZLLbtNAFIanMZcSbgks2VhElcomslGlbiuhSFkWqWkrdaJoZjxOTOammTF1bfwO7BALNizgMXgO3oax46Rxwki2j7//nP+MZg5WLDE2CP4edLwHDx89PnzSffrs+YuXvf6rSyNTTeiESCb1NUaGskTQiU0so9dKU8Qxo1d4+aHSrz5TbRIpLuydolOO5iKJE4KsQ7NeX80gtgtqkX8Msy8wfzfrDYJhUC9/PwibYACadT7rd/7ASJKUU2EJQ8bchIGy0wJpmxBGyy5MDVWILNGc3rhQIE7NtKj3XvpHjkR+LLV7hPVrul1RzDVSi4RkLZ+muoVMiuNknmrXsdVSSM0Ro3xapO7drsEaLaltM8QNR3ZRdo+2qYpi5b6mneuOFruT5HsO5o7jNqw8rZRsxwHzaruC3hLJORJRAYUo3UuKlGOqfQjLtjxyauWFcTEqd7Qx4o1KECvGu3K+JUKcu0uvMiIaQ+VUVV0YYqW/QlnTJi6ydVa+Qfka1S7OjDv3Lb/VSBUroY7LjUSVSZgUjagX8l5yP//D7v4b7KINVvdbXgGrXS+a2eJCr3NG2dZhzQqY+dAk3HdDXydGqN6YG/lwd8D3g8v3wzAYhh9PBmcnzfAfgjfgLTgGITgFZ2AMzsEEEHALfoBf4Lf3yfvqffO+r1I7B03Na9Ba3s9/sRFZug==</latexit><latexit sha1_base64="9UMQ33OoCCIo5NFwcxexC9JqB6k=">AAAD6HicbZLLbtNAFIanMZcSbgks2VhElcomslGlbiuhSFkWqWkrdaJoZjxOTOammTF1bfwO7BALNizgMXgO3oax46Rxwki2j7//nP+MZg5WLDE2CP4edLwHDx89PnzSffrs+YuXvf6rSyNTTeiESCb1NUaGskTQiU0so9dKU8Qxo1d4+aHSrz5TbRIpLuydolOO5iKJE4KsQ7NeX80gtgtqkX8Msy8wfzfrDYJhUC9/PwibYACadT7rd/7ASJKUU2EJQ8bchIGy0wJpmxBGyy5MDVWILNGc3rhQIE7NtKj3XvpHjkR+LLV7hPVrul1RzDVSi4RkLZ+muoVMiuNknmrXsdVSSM0Ro3xapO7drsEaLaltM8QNR3ZRdo+2qYpi5b6mneuOFruT5HsO5o7jNqw8rZRsxwHzaruC3hLJORJRAYUo3UuKlGOqfQjLtjxyauWFcTEqd7Qx4o1KECvGu3K+JUKcu0uvMiIaQ+VUVV0YYqW/QlnTJi6ydVa+Qfka1S7OjDv3Lb/VSBUroY7LjUSVSZgUjagX8l5yP//D7v4b7KINVvdbXgGrXS+a2eJCr3NG2dZhzQqY+dAk3HdDXydGqN6YG/lwd8D3g8v3wzAYhh9PBmcnzfAfgjfgLTgGITgFZ2AMzsEEEHALfoBf4Lf3yfvqffO+r1I7B03Na9Ba3s9/sRFZug==</latexit>

z

<latexit sha1_base64="GmPfnGoOuQrg4mI9U0FrClEwZ5o=">AAAD13icbZLLbtNAFIanMZcSbi0s2VhElVhFNqrEthKKlGVBTVtUR9HMeJyMMjfNjMG1ZbFDLNiwgNfhOXgbxpekdsJIHh9//zn/ORoPUowaGwR/DwbevfsPHh4+Gj5+8vTZ86PjF5dGphqTGZZM6msEDWFUkJmllpFrpQnkiJErtH5f6VefiTZUigt7q8icw6WgCcXQOvQxyhdHo2Ac1MvfD8I2GIF2nS+OB3+iWOKUE2Exg8bchIGy8wJqSzEj5TBKDVEQr+GS3LhQQE7MvKhHLf0TR2I/kdo9wvo17VYUSw3ViuKs59NW95BJUUKXqXYdey2F1BwywudF6vZ+DdJwTWyfQW44tKtyeNKlKk6Ue5t+rjtJ5A6O7zmYW476sPK0UrIdB8SrcQX5giXnUMRFJETpNilSjoj2o6jsyxOnVl4IFZNyR5tC3qoYsmK6K+cdMUI5sbDKiEkSKaeq6odBVvoNyto2SZFtsvItyjeodnFm3Ll3/JBdVbwR6rjcSkQZyqRoRb2Sd5L7+B92/7/FLtpidTdyA6x2vUhmiwu9yZlkncNaFFHmR4ZyXy2axBjWg7krH+5e8P3g8u04DMbhh9PR2Wl7+Q/BK/AavAEheAfOwBScgxnAIAE/wC/w2/vkffW+ed+b1MFBW/MS9Jb38x/u7VOP</latexit><latexit sha1_base64="GmPfnGoOuQrg4mI9U0FrClEwZ5o=">AAAD13icbZLLbtNAFIanMZcSbi0s2VhElVhFNqrEthKKlGVBTVtUR9HMeJyMMjfNjMG1ZbFDLNiwgNfhOXgbxpekdsJIHh9//zn/ORoPUowaGwR/DwbevfsPHh4+Gj5+8vTZ86PjF5dGphqTGZZM6msEDWFUkJmllpFrpQnkiJErtH5f6VefiTZUigt7q8icw6WgCcXQOvQxyhdHo2Ac1MvfD8I2GIF2nS+OB3+iWOKUE2Exg8bchIGy8wJqSzEj5TBKDVEQr+GS3LhQQE7MvKhHLf0TR2I/kdo9wvo17VYUSw3ViuKs59NW95BJUUKXqXYdey2F1BwywudF6vZ+DdJwTWyfQW44tKtyeNKlKk6Ue5t+rjtJ5A6O7zmYW476sPK0UrIdB8SrcQX5giXnUMRFJETpNilSjoj2o6jsyxOnVl4IFZNyR5tC3qoYsmK6K+cdMUI5sbDKiEkSKaeq6odBVvoNyto2SZFtsvItyjeodnFm3Ll3/JBdVbwR6rjcSkQZyqRoRb2Sd5L7+B92/7/FLtpidTdyA6x2vUhmiwu9yZlkncNaFFHmR4ZyXy2axBjWg7krH+5e8P3g8u04DMbhh9PR2Wl7+Q/BK/AavAEheAfOwBScgxnAIAE/wC/w2/vkffW+ed+b1MFBW/MS9Jb38x/u7VOP</latexit><latexit sha1_base64="GmPfnGoOuQrg4mI9U0FrClEwZ5o=">AAAD13icbZLLbtNAFIanMZcSbi0s2VhElVhFNqrEthKKlGVBTVtUR9HMeJyMMjfNjMG1ZbFDLNiwgNfhOXgbxpekdsJIHh9//zn/ORoPUowaGwR/DwbevfsPHh4+Gj5+8vTZ86PjF5dGphqTGZZM6msEDWFUkJmllpFrpQnkiJErtH5f6VefiTZUigt7q8icw6WgCcXQOvQxyhdHo2Ac1MvfD8I2GIF2nS+OB3+iWOKUE2Exg8bchIGy8wJqSzEj5TBKDVEQr+GS3LhQQE7MvKhHLf0TR2I/kdo9wvo17VYUSw3ViuKs59NW95BJUUKXqXYdey2F1BwywudF6vZ+DdJwTWyfQW44tKtyeNKlKk6Ue5t+rjtJ5A6O7zmYW476sPK0UrIdB8SrcQX5giXnUMRFJETpNilSjoj2o6jsyxOnVl4IFZNyR5tC3qoYsmK6K+cdMUI5sbDKiEkSKaeq6odBVvoNyto2SZFtsvItyjeodnFm3Ll3/JBdVbwR6rjcSkQZyqRoRb2Sd5L7+B92/7/FLtpidTdyA6x2vUhmiwu9yZlkncNaFFHmR4ZyXy2axBjWg7krH+5e8P3g8u04DMbhh9PR2Wl7+Q/BK/AavAEheAfOwBScgxnAIAE/wC/w2/vkffW+ed+b1MFBW/MS9Jb38x/u7VOP</latexit><latexit sha1_base64="GmPfnGoOuQrg4mI9U0FrClEwZ5o=">AAAD13icbZLLbtNAFIanMZcSbi0s2VhElVhFNqrEthKKlGVBTVtUR9HMeJyMMjfNjMG1ZbFDLNiwgNfhOXgbxpekdsJIHh9//zn/ORoPUowaGwR/DwbevfsPHh4+Gj5+8vTZ86PjF5dGphqTGZZM6msEDWFUkJmllpFrpQnkiJErtH5f6VefiTZUigt7q8icw6WgCcXQOvQxyhdHo2Ac1MvfD8I2GIF2nS+OB3+iWOKUE2Exg8bchIGy8wJqSzEj5TBKDVEQr+GS3LhQQE7MvKhHLf0TR2I/kdo9wvo17VYUSw3ViuKs59NW95BJUUKXqXYdey2F1BwywudF6vZ+DdJwTWyfQW44tKtyeNKlKk6Ue5t+rjtJ5A6O7zmYW476sPK0UrIdB8SrcQX5giXnUMRFJETpNilSjoj2o6jsyxOnVl4IFZNyR5tC3qoYsmK6K+cdMUI5sbDKiEkSKaeq6odBVvoNyto2SZFtsvItyjeodnFm3Ll3/JBdVbwR6rjcSkQZyqRoRb2Sd5L7+B92/7/FLtpidTdyA6x2vUhmiwu9yZlkncNaFFHmR4ZyXy2axBjWg7krH+5e8P3g8u04DMbhh9PR2Wl7+Q/BK/AavAEheAfOwBScgxnAIAE/wC/w2/vkffW+ed+b1MFBW/MS9Jb38x/u7VOP</latexit>

p

✓

(

z

)

<latexit sha1_base64="8OB4rJ6bykF0LWQXTLeGxuBuoRA=">AAAD53icbZLdatswFMfVeB9d9pVul7sxC4XuJtij0NvCCOSyg6Yt1CFIspyI6AtJ7twIP8Puxi52s4vtNfYce5vJsZPGyQS2j3//c/5HSAcpRo2Nor8HneDR4ydPD591n794+ep17+jNlZG5xmSMJZP6BkFDGBVkbKll5EZpAjli5BotPlX69R3Rhkpxae8VmXA4EzSjGFqPpr2emroE2TmxsAxPkuWHaa8fDaLVCveDuAn6oFkX06POnySVOOdEWMygMbdxpOzEQW0pZqTsJrkhCuIFnJFbHwrIiZm41dbL8NiTNMyk9o+w4YpuV7iZhmpOcdHyaapbyOQoo7Nc+46tlkJqDhnhE5f7d7sGabggts0gNxzaedk93qYqzZT/mnauP1nkD5LvOZh7jtqw8rRSsh0HxKvtCvIFS86hSF0iROlfUuQcER0mSdmWh16tvBByw3JHG0HeqBgyN9qVl1tigpbVlfuMlGSJ8qqqLgyyMqxR0bTJXLHOWm7Qco1WLt6Me/ctv3qgXC3Uw7WRiDKUSdGIei4fJP/zP+zvv8E+2mD1sOUaWO17kcK6S73OGRZbh+UHvQgTQ3mopnViClcb8yMf7w74fnD1cRBHg/jzaf/8tBn+Q/AOvAcnIAZn4ByMwAUYAwzuwA/wC/wOaPA1+BZ8r1M7B03NW9Bawc9/VwRZWA==</latexit><latexit sha1_base64="8OB4rJ6bykF0LWQXTLeGxuBuoRA=">AAAD53icbZLdatswFMfVeB9d9pVul7sxC4XuJtij0NvCCOSyg6Yt1CFIspyI6AtJ7twIP8Puxi52s4vtNfYce5vJsZPGyQS2j3//c/5HSAcpRo2Nor8HneDR4ydPD591n794+ep17+jNlZG5xmSMJZP6BkFDGBVkbKll5EZpAjli5BotPlX69R3Rhkpxae8VmXA4EzSjGFqPpr2emroE2TmxsAxPkuWHaa8fDaLVCveDuAn6oFkX06POnySVOOdEWMygMbdxpOzEQW0pZqTsJrkhCuIFnJFbHwrIiZm41dbL8NiTNMyk9o+w4YpuV7iZhmpOcdHyaapbyOQoo7Nc+46tlkJqDhnhE5f7d7sGabggts0gNxzaedk93qYqzZT/mnauP1nkD5LvOZh7jtqw8rRSsh0HxKvtCvIFS86hSF0iROlfUuQcER0mSdmWh16tvBByw3JHG0HeqBgyN9qVl1tigpbVlfuMlGSJ8qqqLgyyMqxR0bTJXLHOWm7Qco1WLt6Me/ctv3qgXC3Uw7WRiDKUSdGIei4fJP/zP+zvv8E+2mD1sOUaWO17kcK6S73OGRZbh+UHvQgTQ3mopnViClcb8yMf7w74fnD1cRBHg/jzaf/8tBn+Q/AOvAcnIAZn4ByMwAUYAwzuwA/wC/wOaPA1+BZ8r1M7B03NW9Bawc9/VwRZWA==</latexit><latexit sha1_base64="8OB4rJ6bykF0LWQXTLeGxuBuoRA=">AAAD53icbZLdatswFMfVeB9d9pVul7sxC4XuJtij0NvCCOSyg6Yt1CFIspyI6AtJ7twIP8Puxi52s4vtNfYce5vJsZPGyQS2j3//c/5HSAcpRo2Nor8HneDR4ydPD591n794+ep17+jNlZG5xmSMJZP6BkFDGBVkbKll5EZpAjli5BotPlX69R3Rhkpxae8VmXA4EzSjGFqPpr2emroE2TmxsAxPkuWHaa8fDaLVCveDuAn6oFkX06POnySVOOdEWMygMbdxpOzEQW0pZqTsJrkhCuIFnJFbHwrIiZm41dbL8NiTNMyk9o+w4YpuV7iZhmpOcdHyaapbyOQoo7Nc+46tlkJqDhnhE5f7d7sGabggts0gNxzaedk93qYqzZT/mnauP1nkD5LvOZh7jtqw8rRSsh0HxKvtCvIFS86hSF0iROlfUuQcER0mSdmWh16tvBByw3JHG0HeqBgyN9qVl1tigpbVlfuMlGSJ8qqqLgyyMqxR0bTJXLHOWm7Qco1WLt6Me/ctv3qgXC3Uw7WRiDKUSdGIei4fJP/zP+zvv8E+2mD1sOUaWO17kcK6S73OGRZbh+UHvQgTQ3mopnViClcb8yMf7w74fnD1cRBHg/jzaf/8tBn+Q/AOvAcnIAZn4ByMwAUYAwzuwA/wC/wOaPA1+BZ8r1M7B03NW9Bawc9/VwRZWA==</latexit><latexit sha1_base64="8OB4rJ6bykF0LWQXTLeGxuBuoRA=">AAAD53icbZLdatswFMfVeB9d9pVul7sxC4XuJtij0NvCCOSyg6Yt1CFIspyI6AtJ7twIP8Puxi52s4vtNfYce5vJsZPGyQS2j3//c/5HSAcpRo2Nor8HneDR4ydPD591n794+ep17+jNlZG5xmSMJZP6BkFDGBVkbKll5EZpAjli5BotPlX69R3Rhkpxae8VmXA4EzSjGFqPpr2emroE2TmxsAxPkuWHaa8fDaLVCveDuAn6oFkX06POnySVOOdEWMygMbdxpOzEQW0pZqTsJrkhCuIFnJFbHwrIiZm41dbL8NiTNMyk9o+w4YpuV7iZhmpOcdHyaapbyOQoo7Nc+46tlkJqDhnhE5f7d7sGabggts0gNxzaedk93qYqzZT/mnauP1nkD5LvOZh7jtqw8rRSsh0HxKvtCvIFS86hSF0iROlfUuQcER0mSdmWh16tvBByw3JHG0HeqBgyN9qVl1tigpbVlfuMlGSJ8qqqLgyyMqxR0bTJXLHOWm7Qco1WLt6Me/ctv3qgXC3Uw7WRiDKUSdGIei4fJP/zP+zvv8E+2mD1sOUaWO17kcK6S73OGRZbh+UHvQgTQ3mopnViClcb8yMf7w74fnD1cRBHg/jzaf/8tBn+Q/AOvAcnIAZn4ByMwAUYAwzuwA/wC/wOaPA1+BZ8r1M7B03NW9Bawc9/VwRZWA==</latexit>

[Benedetti et al., QST 3]

Invertible Flows

Generative Adversarial

Networks (GAN)

Variational Autoencoders

(VAE)

[Wilson et al.,

(8)

• Implemented a

deep convolutional VAE with discrete latent space

-

288-dimensional latent space

-

Prior is a Chimera-structured (C6) RBM

• Model

trained end-to-end

using only

samples obtained from the quantum

annealer

• Successful training validated by estimating

the log-likelihood of the model

-

Showed

improvement from a trivial

classical baseline

(Bernoulli)

-

Match performance of model trained

with Population Annealing (PA)

[Trained on MNIST]

[Samples generated with D-Wave 2000Q]

[state-of-the-art: LL~-79.5]

[Khoshaman, Amin, NIPS 2018]

MNIST (dynamic binarization)

LL

Sampler

Chimera

Bernoulli

DW2000Q

82 .

8 _±

0 .

2

83 .

7 _±

0 .

2 PA

82 .

8 _±

0 .

1

84 .

2 _±

0 .

05

(9)

• Larger RBM

-

Model building to improve use of physical connectivities

-

Develop denser physical connectivities

-

More complex datasets

• Multi-modality

-

Latent-space RBM must develop multi-modal distributions

• Sampling reliability

-

Reduce control errors for more reliable training

Reliably

sample from

large RBM

, representing complex

(10)

Latent-space

exploitation with VAEs

ELBO

=

E

x

⇠

p

data



E

⇣

⇠

q

(⇣

|

x

)



log

p

✓

(

x

|

⇣

)

log

q

(

⇣

_|

x

)

log

p✓

(

⇣

)

<latexit sha1_base64="cKc+5V1rKPfZrLMo99TE5LPUI4Q=">AAAEhHicbVPdbtMwGE1LGaP8deySG4tq0iZB1YwhxAVo2lSpF0gMad0m1VVlp04b1X/YDmT18kg8A8/B2+D8tGvaWUr8+ZzzHTv+vmBJI2263X+1+qPG450nu0+bz56/ePmqtff6SotYBWQQCCrUDUaa0IiTgYkMJTdSEcQwJdd4fp7x17+I0pHgl+ZWkhFDUx6FUYCMg8atPxYqBiAOQe/b2fcUfAGwl0BKQjMELhxbiBfEIKgjBn5mKzmL0sMCvIPJUVpoIRVTIDPezBzjFMldIToC4D2AoUKBzUUPmtjt/CI5hSqazsyonMatdrfTzQfYDvwyaHvluBjv1f/CiQhiRrgJKNJ66HelGVmkTBRQkjZhrIlEwRxNydCFHDGiRza/1xQcOGQCQqHcww3I0fUMO1XIfUqQVHzK7AqkYxxG01i5HStbcqEYooSNbOze1Rys0JyYKoaYZsjM0ubBOionoXSzrmpd2bGrMtty0LcMV8HM0whBNxwwy47Lye9AMIb4xELOXa244DHDRAEI0yrdc2zmhbHtpRtcH7GSDRC1/U16sUYWxc8UExJC6ViZFQzRFBRQUm4T2mSpWqygxRLKXZwZc+5rfkWL2YIo2m1FEakjKnhJqpm4p9ziIdjVv4Szpl6d+f7IBWCU24skxl6qpaaXrF2Wa/0E5H+ZHBfCCcoP5lre32zw7eDquON3O/6Pk/bpSdn8u94b76136PneJ+/U63sX3sALavu1z7Wz2nljp/Gu8aHxsZDWa2XOvlcZja//AdEokUA=</latexit><latexit sha1_base64="cKc+5V1rKPfZrLMo99TE5LPUI4Q=">AAAEhHicbVPdbtMwGE1LGaP8deySG4tq0iZB1YwhxAVo2lSpF0gMad0m1VVlp04b1X/YDmT18kg8A8/B2+D8tGvaWUr8+ZzzHTv+vmBJI2263X+1+qPG450nu0+bz56/ePmqtff6SotYBWQQCCrUDUaa0IiTgYkMJTdSEcQwJdd4fp7x17+I0pHgl+ZWkhFDUx6FUYCMg8atPxYqBiAOQe/b2fcUfAGwl0BKQjMELhxbiBfEIKgjBn5mKzmL0sMCvIPJUVpoIRVTIDPezBzjFMldIToC4D2AoUKBzUUPmtjt/CI5hSqazsyonMatdrfTzQfYDvwyaHvluBjv1f/CiQhiRrgJKNJ66HelGVmkTBRQkjZhrIlEwRxNydCFHDGiRza/1xQcOGQCQqHcww3I0fUMO1XIfUqQVHzK7AqkYxxG01i5HStbcqEYooSNbOze1Rys0JyYKoaYZsjM0ubBOionoXSzrmpd2bGrMtty0LcMV8HM0whBNxwwy47Lye9AMIb4xELOXa244DHDRAEI0yrdc2zmhbHtpRtcH7GSDRC1/U16sUYWxc8UExJC6ViZFQzRFBRQUm4T2mSpWqygxRLKXZwZc+5rfkWL2YIo2m1FEakjKnhJqpm4p9ziIdjVv4Szpl6d+f7IBWCU24skxl6qpaaXrF2Wa/0E5H+ZHBfCCcoP5lre32zw7eDquON3O/6Pk/bpSdn8u94b76136PneJ+/U63sX3sALavu1z7Wz2nljp/Gu8aHxsZDWa2XOvlcZja//AdEokUA=</latexit><latexit sha1_base64="cKc+5V1rKPfZrLMo99TE5LPUI4Q=">AAAEhHicbVPdbtMwGE1LGaP8deySG4tq0iZB1YwhxAVo2lSpF0gMad0m1VVlp04b1X/YDmT18kg8A8/B2+D8tGvaWUr8+ZzzHTv+vmBJI2263X+1+qPG450nu0+bz56/ePmqtff6SotYBWQQCCrUDUaa0IiTgYkMJTdSEcQwJdd4fp7x17+I0pHgl+ZWkhFDUx6FUYCMg8atPxYqBiAOQe/b2fcUfAGwl0BKQjMELhxbiBfEIKgjBn5mKzmL0sMCvIPJUVpoIRVTIDPezBzjFMldIToC4D2AoUKBzUUPmtjt/CI5hSqazsyonMatdrfTzQfYDvwyaHvluBjv1f/CiQhiRrgJKNJ66HelGVmkTBRQkjZhrIlEwRxNydCFHDGiRza/1xQcOGQCQqHcww3I0fUMO1XIfUqQVHzK7AqkYxxG01i5HStbcqEYooSNbOze1Rys0JyYKoaYZsjM0ubBOionoXSzrmpd2bGrMtty0LcMV8HM0whBNxwwy47Lye9AMIb4xELOXa244DHDRAEI0yrdc2zmhbHtpRtcH7GSDRC1/U16sUYWxc8UExJC6ViZFQzRFBRQUm4T2mSpWqygxRLKXZwZc+5rfkWL2YIo2m1FEakjKnhJqpm4p9ziIdjVv4Szpl6d+f7IBWCU24skxl6qpaaXrF2Wa/0E5H+ZHBfCCcoP5lre32zw7eDquON3O/6Pk/bpSdn8u94b76136PneJ+/U63sX3sALavu1z7Wz2nljp/Gu8aHxsZDWa2XOvlcZja//AdEokUA=</latexit><latexit sha1_base64="cKc+5V1rKPfZrLMo99TE5LPUI4Q=">AAAEhHicbVPdbtMwGE1LGaP8deySG4tq0iZB1YwhxAVo2lSpF0gMad0m1VVlp04b1X/YDmT18kg8A8/B2+D8tGvaWUr8+ZzzHTv+vmBJI2263X+1+qPG450nu0+bz56/ePmqtff6SotYBWQQCCrUDUaa0IiTgYkMJTdSEcQwJdd4fp7x17+I0pHgl+ZWkhFDUx6FUYCMg8atPxYqBiAOQe/b2fcUfAGwl0BKQjMELhxbiBfEIKgjBn5mKzmL0sMCvIPJUVpoIRVTIDPezBzjFMldIToC4D2AoUKBzUUPmtjt/CI5hSqazsyonMatdrfTzQfYDvwyaHvluBjv1f/CiQhiRrgJKNJ66HelGVmkTBRQkjZhrIlEwRxNydCFHDGiRza/1xQcOGQCQqHcww3I0fUMO1XIfUqQVHzK7AqkYxxG01i5HStbcqEYooSNbOze1Rys0JyYKoaYZsjM0ubBOionoXSzrmpd2bGrMtty0LcMV8HM0whBNxwwy47Lye9AMIb4xELOXa244DHDRAEI0yrdc2zmhbHtpRtcH7GSDRC1/U16sUYWxc8UExJC6ViZFQzRFBRQUm4T2mSpWqygxRLKXZwZc+5rfkWL2YIo2m1FEakjKnhJqpm4p9ziIdjVv4Szpl6d+f7IBWCU24skxl6qpaaXrF2Wa/0E5H+ZHBfCCcoP5lre32zw7eDquON3O/6Pk/bpSdn8u94b76136PneJ+/U63sX3sALavu1z7Wz2nljp/Gu8aHxsZDWa2XOvlcZja//AdEokUA=</latexit>