LanceDB CRAG With Langgraph

CRAG With Langgraph

agentsllmsvector-databaseCorrective-RAG-with_LanggraphlancedbgptopenaiAImultimodal-aitutorialsmachine-learningembeddingsfine-tuningdeep-learninggpt-4-visionllama-indexragmultimodallangchainlancedb-recipes

alph-notebooks/lancedb-recipes / CRAG_with_Langgraph.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

Corrective RAG(CRAG)

Self-reflection can enhance RAG, enabling correction of poor quality retrieval or generations.

Corrective Retrieval-Augmented Generation (CRAG) is a method that works like a built-in fact-checker.

It adds both creativity and accuracy by creating text and then checking for any mistakes or made-up information. This helps make sure the final result is reliable and matches real-world facts. It's like a safety feature for AI writers, making their work more trustworthy and lowering the chances of spreading false information.

Here we'll see how to implement ideas from the Corrective RAG (CRAG) Paper using LangGraph.

[ ]

Set Tavily Search API Key for web search and OpenAI key

Tavily is a search engine specifically designed for AI agents and large language models (LLMs).

Get free credits link

[ ]

Corrective-RAG (CRAG) is a recent paper that talks about a cool way to make a self-reflective RAG.

The method givesating/scores retrieved documents based on how well they answer a question:

For Correct documents -

If at least one document is really relevant, it moves on to creating text
Before creating text, it cleans up the knowledge
This breaks down the document into "knowledge strips"
It rates each strip and gets rid of ones that don't matter

For Ambiguous or Incorrect documents -

If all documents are not relevant enough or if it's not sure, the method looks for more information
It uses a web search to add more details to what it found
The diagram in the paper also show that they might change the question to get better results.

Building Retriever

[ ]

Defining Langgraph

We will define a graph for building Langgraph.

We can access any Graph node as state['keys'].

[ ]

Defining Nodes for Langgraph

Nodes and Edges

Each Node will modify the state.

Each Edge will choose which node to call next.

Next Steps:

Retrieve Relevant Documents
If Relevant Document not found, Go for Supplement Retrieval with Web search(using Tavily API).
Query Re-writing to optimize the query for Web search.

Here is our graph flow:

[ ]

Graph Edges

[ ]

Build Graph

Follow the flow we outlined in the figure above.

[ ]

*****  RETRIEVE  *****
{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),
                 Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),
                 Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nJay Alammar\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\n\n\nBlog\nAbout\n\n\n\n\n\n\nThe Illustrated Transformer\n\nDiscussions:\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\n\n\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\n\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),
                 Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],
  'question': 'How Transformers work?'}
'-------'
*****  DOCS RELEVANCE CHECK *****
*****  RATED DOCUMENT: RELEVANT *****
*****  RATED DOCUMENT: RELEVANT *****
*****  RATED DOCUMENT: RELEVANT *****
*****  RATED DOCUMENT: RELEVANT *****
{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),
                 Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),
                 Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nJay Alammar\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\n\n\nBlog\nAbout\n\n\n\n\n\n\nThe Illustrated Transformer\n\nDiscussions:\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\n\n\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\n\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),
                 Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],
  'question': 'How Transformers work?',
  'run_web_search': 'No'}
'-------'
*****  DECIDE TO GENERATE  *****
*****  DECISION: GENERATE  *****
*****  GENERATE  *****
{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),
                 Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),
                 Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nJay Alammar\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\n\n\nBlog\nAbout\n\n\n\n\n\n\nThe Illustrated Transformer\n\nDiscussions:\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\n\n\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\n\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),
                 Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],
  'generation': 'Transformers work by using attention mechanisms to boost the '
                'speed of training deep learning models. They outperform '
                'previous models like the Google Neural Machine Translation '
                'model in specific tasks, mainly due to their ability to '
                'parallelize operations, making them highly efficient for '
                'training on modern hardware. Transformers use self-attention '
                'to weigh the significance of different words in a sentence, '
                'allowing the model to focus on relevant words and ignore less '
                'relevant ones, improving the performance of tasks like '
                'translation and text summarization.',
  'question': 'How Transformers work?'}
'-------'
{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),
                 Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),
                 Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nJay Alammar\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\n\n\nBlog\nAbout\n\n\n\n\n\n\nThe Illustrated Transformer\n\nDiscussions:\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\n\n\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\n\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),
                 Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],
  'generation': 'Transformers work by using attention mechanisms to boost the '
                'speed of training deep learning models. They outperform '
                'previous models like the Google Neural Machine Translation '
                'model in specific tasks, mainly due to their ability to '
                'parallelize operations, making them highly efficient for '
                'training on modern hardware. Transformers use self-attention '
                'to weigh the significance of different words in a sentence, '
                'allowing the model to focus on relevant words and ignore less '
                'relevant ones, improving the performance of tasks like '
                'translation and text summarization.',
  'question': 'How Transformers work?'}
'-------'
*****  Generated Answer  *****
('Transformers work by using attention mechanisms to boost the speed of '
 'training deep learning models. They outperform previous models like the '
 'Google Neural Machine Translation model in specific tasks, mainly due to '
 'their ability to parallelize operations, making them highly efficient for '
 'training on modern hardware. Transformers use self-attention to weigh the '
 'significance of different words in a sentence, allowing the model to focus '
 'on relevant words and ignore less relevant ones, improving the performance '
 'of tasks like translation and text summarization.')

[ ]

*****  RETRIEVE  *****
{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3664601147174835}),
                 Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nJay Alammar\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\n\n\nBlog\nAbout\n\n\n\n\n\n\nThe Illustrated Transformer\n\nDiscussions:\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\n\n\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\n\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3938315510749817}),
                 Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.40166381001472473}),
                 Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.41414153575897217})],
  'question': 'What is Vision transformer and How it is different from normal '
              'transformers?'}
'------------------------'
*****  DOCS RELEVANCE CHECK *****
*****  RATED DOCUMENT: NOT RELEVANT *****
*****  RATED DOCUMENT: NOT RELEVANT *****
*****  RATED DOCUMENT: NOT RELEVANT *****
*****  RATED DOCUMENT: NOT RELEVANT *****
{ 'documents': [],
  'question': 'What is Vision transformer and How it is different from normal '
              'transformers?',
  'run_web_search': 'Yes'}
'------------------------'
*****  DECIDE TO GENERATE  *****
*****  DECISION: TRANSFORM QUERY and RUN WEB SEARCH  *****
***** TRANSFORM QUERY *****
{ 'documents': [],
  'question': 'What distinguishes a Vision Transformer from traditional '
              'Transformer models in machine learning?'}
'------------------------'
*****  WEB SEARCH  *****
{ 'documents': [ Document(page_content="Deep Learning System 2\nAt the 2019 Neural Information Processing Systems Conference (NeurIPS 2019), Yoshua Bengio, one of the three pioneers of deep learning, gave a keynote speech that shed light on the possible transition of System 1 deep learning to System 2.\nTwo systems of thinking. Integrating Deep Learning with other technologies\nExtended Natural Language Processing Capabilities\nIncreased Attention to Ethical Considerations\nIntegration of Hybrid Models\nSuch deep learning trends in 2024 will drive the scalability, popularity, and creation of more effective approaches in the field. If you want to build a technology product using deep learning, Merehead is your best choice!\nSelf-supervised Learning\nSelf-supervised learning is a paradigm in which a model learns to extract meaningful representations or features from unlabeled data without the need for explicit labels provided by humans.\n Deep learning models successfully solve tasks such as:\nDeep learning trends in 2024 may increase the number of significant advances and breakthroughs due to the availability of big data, computational resources, and advanced algorithms.\n How have deep learning trends shaped?\nUnderstanding how deep learning trends have formed in the past can give us a better idea of what we can expect from this field in the future.\nTrain\nDevelop\nDeploy\nOperate\nData Collection\nBuilding Blocks\u200b\nDevice Enrollment\nMonitoring Dashboards\nVideo Annotation\u200b\nApplication Editor\u200b\nDevice Management\nRemote Maintenance\nModel Training\nApplication Library\nDeployment Manager\nUnified Security Center\nAI Model Library\nConfiguration Manager\nIoT Edge Gateway\nPrivacy-preserving AI\nReady to get started?\nVision Transformers (ViT) in Image Recognition – 2024 Guide\nViso Suite is the all-in-one solution for teams to build, deliver, scale computer vision applications.\n The self-attention layer calculates attention weights for each pixel in the image based on its relationship with all other pixels, while the feed-forward layer applies a non-linear transformation to the output of the self-attention layer. The final output of the ViT architecture is a class prediction, obtained by passing the output of the last transformer block through a classification head, which typically consists of a single fully connected layer.\n The model also learns from training data to encode the relative location of the image patches to reconstruct the structure of the image.\n The brighter the color of a pixel in the heatmap, the higher the attention weight between the corresponding tokens.\nAfter their initial success in natural language processing, transformer ar-chitectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmenta-tion, and video analysis.\nThere are multiple blocks in the ViT encoder, and each block consists of three major processing elements:\nLayer Norm keeps the training process on track and lets the model adapt to the variations among the training images.\n The (single-head) attention mechanism in the above figure uses Q and K. Still, in the multi-head attention mechanism, each head has its projection matrix W_i^Q, W_i^K, and W_i^V, and they calculate the attention weights using the feature values projected using these matrices.\n ViT vs. Convolutional Neural Networks\nThe ViT model represents an input image as a series of image patches, like the series of word embeddings used when using transformers to text, and directly predicts class labels for the image.\n Transformer Architecture modified for images (ViT)\n\u200d\nThe paper suggests using a Transformer Encoder as a base model to extract features from the image and passing these “processed” features into a Multilayer Perceptron (MLP) head model for classification.\n Although transformer-based models are known to only be effective when large training datasets are available, we show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets\nAutonomous driving\nOn Tesla AI Day in 2021, Tesla revealed many intricate inner workings of the neural network powering Tesla FSD.\nThere has been a dramatic increase in the complexity of the computer vision model landscape. Many models are now at your fingertips, from the first ConvNets to the latest Vision Transformers. Just as supervised learning on ImageNet gave way to self-supervised learning and image-text pair training, such as CLIP, so did training paradigms. Compared to ResNets, CLIP's visual encoder is far more ...")],
  'question': 'What distinguishes a Vision Transformer from traditional '
              'Transformer models in machine learning?'}
'------------------------'
*****  GENERATE  *****
{ 'documents': [ Document(page_content="Deep Learning System 2\nAt the 2019 Neural Information Processing Systems Conference (NeurIPS 2019), Yoshua Bengio, one of the three pioneers of deep learning, gave a keynote speech that shed light on the possible transition of System 1 deep learning to System 2.\nTwo systems of thinking. Integrating Deep Learning with other technologies\nExtended Natural Language Processing Capabilities\nIncreased Attention to Ethical Considerations\nIntegration of Hybrid Models\nSuch deep learning trends in 2024 will drive the scalability, popularity, and creation of more effective approaches in the field. If you want to build a technology product using deep learning, Merehead is your best choice!\nSelf-supervised Learning\nSelf-supervised learning is a paradigm in which a model learns to extract meaningful representations or features from unlabeled data without the need for explicit labels provided by humans.\n Deep learning models successfully solve tasks such as:\nDeep learning trends in 2024 may increase the number of significant advances and breakthroughs due to the availability of big data, computational resources, and advanced algorithms.\n How have deep learning trends shaped?\nUnderstanding how deep learning trends have formed in the past can give us a better idea of what we can expect from this field in the future.\nTrain\nDevelop\nDeploy\nOperate\nData Collection\nBuilding Blocks\u200b\nDevice Enrollment\nMonitoring Dashboards\nVideo Annotation\u200b\nApplication Editor\u200b\nDevice Management\nRemote Maintenance\nModel Training\nApplication Library\nDeployment Manager\nUnified Security Center\nAI Model Library\nConfiguration Manager\nIoT Edge Gateway\nPrivacy-preserving AI\nReady to get started?\nVision Transformers (ViT) in Image Recognition – 2024 Guide\nViso Suite is the all-in-one solution for teams to build, deliver, scale computer vision applications.\n The self-attention layer calculates attention weights for each pixel in the image based on its relationship with all other pixels, while the feed-forward layer applies a non-linear transformation to the output of the self-attention layer. The final output of the ViT architecture is a class prediction, obtained by passing the output of the last transformer block through a classification head, which typically consists of a single fully connected layer.\n The model also learns from training data to encode the relative location of the image patches to reconstruct the structure of the image.\n The brighter the color of a pixel in the heatmap, the higher the attention weight between the corresponding tokens.\nAfter their initial success in natural language processing, transformer ar-chitectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmenta-tion, and video analysis.\nThere are multiple blocks in the ViT encoder, and each block consists of three major processing elements:\nLayer Norm keeps the training process on track and lets the model adapt to the variations among the training images.\n The (single-head) attention mechanism in the above figure uses Q and K. Still, in the multi-head attention mechanism, each head has its projection matrix W_i^Q, W_i^K, and W_i^V, and they calculate the attention weights using the feature values projected using these matrices.\n ViT vs. Convolutional Neural Networks\nThe ViT model represents an input image as a series of image patches, like the series of word embeddings used when using transformers to text, and directly predicts class labels for the image.\n Transformer Architecture modified for images (ViT)\n\u200d\nThe paper suggests using a Transformer Encoder as a base model to extract features from the image and passing these “processed” features into a Multilayer Perceptron (MLP) head model for classification.\n Although transformer-based models are known to only be effective when large training datasets are available, we show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets\nAutonomous driving\nOn Tesla AI Day in 2021, Tesla revealed many intricate inner workings of the neural network powering Tesla FSD.\nThere has been a dramatic increase in the complexity of the computer vision model landscape. Many models are now at your fingertips, from the first ConvNets to the latest Vision Transformers. Just as supervised learning on ImageNet gave way to self-supervised learning and image-text pair training, such as CLIP, so did training paradigms. Compared to ResNets, CLIP's visual encoder is far more ...")],
  'generation': 'A Vision Transformer (ViT) distinguishes itself from '
                'traditional Transformer models by applying the transformer '
                'architecture to image recognition tasks, where it treats an '
                'input image as a series of image patches similar to how text '
                'transformers treat a series of word embeddings. Unlike '
                'traditional models that might rely on convolutional layers, '
                'ViTs use self-attention mechanisms to calculate attention '
                'weights for each pixel in relation to all other pixels, '
                'enabling it to directly predict class labels for the image. '
                'This approach allows for the encoding of the relative '
                "location of image patches to reconstruct the image's "
                "structure, leveraging the transformer's strengths in handling "
                'sequential data for computer vision tasks.',
  'question': 'What distinguishes a Vision Transformer from traditional '
              'Transformer models in machine learning?'}
'------------------------'
{ 'documents': [ Document(page_content="Deep Learning System 2\nAt the 2019 Neural Information Processing Systems Conference (NeurIPS 2019), Yoshua Bengio, one of the three pioneers of deep learning, gave a keynote speech that shed light on the possible transition of System 1 deep learning to System 2.\nTwo systems of thinking. Integrating Deep Learning with other technologies\nExtended Natural Language Processing Capabilities\nIncreased Attention to Ethical Considerations\nIntegration of Hybrid Models\nSuch deep learning trends in 2024 will drive the scalability, popularity, and creation of more effective approaches in the field. If you want to build a technology product using deep learning, Merehead is your best choice!\nSelf-supervised Learning\nSelf-supervised learning is a paradigm in which a model learns to extract meaningful representations or features from unlabeled data without the need for explicit labels provided by humans.\n Deep learning models successfully solve tasks such as:\nDeep learning trends in 2024 may increase the number of significant advances and breakthroughs due to the availability of big data, computational resources, and advanced algorithms.\n How have deep learning trends shaped?\nUnderstanding how deep learning trends have formed in the past can give us a better idea of what we can expect from this field in the future.\nTrain\nDevelop\nDeploy\nOperate\nData Collection\nBuilding Blocks\u200b\nDevice Enrollment\nMonitoring Dashboards\nVideo Annotation\u200b\nApplication Editor\u200b\nDevice Management\nRemote Maintenance\nModel Training\nApplication Library\nDeployment Manager\nUnified Security Center\nAI Model Library\nConfiguration Manager\nIoT Edge Gateway\nPrivacy-preserving AI\nReady to get started?\nVision Transformers (ViT) in Image Recognition – 2024 Guide\nViso Suite is the all-in-one solution for teams to build, deliver, scale computer vision applications.\n The self-attention layer calculates attention weights for each pixel in the image based on its relationship with all other pixels, while the feed-forward layer applies a non-linear transformation to the output of the self-attention layer. The final output of the ViT architecture is a class prediction, obtained by passing the output of the last transformer block through a classification head, which typically consists of a single fully connected layer.\n The model also learns from training data to encode the relative location of the image patches to reconstruct the structure of the image.\n The brighter the color of a pixel in the heatmap, the higher the attention weight between the corresponding tokens.\nAfter their initial success in natural language processing, transformer ar-chitectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmenta-tion, and video analysis.\nThere are multiple blocks in the ViT encoder, and each block consists of three major processing elements:\nLayer Norm keeps the training process on track and lets the model adapt to the variations among the training images.\n The (single-head) attention mechanism in the above figure uses Q and K. Still, in the multi-head attention mechanism, each head has its projection matrix W_i^Q, W_i^K, and W_i^V, and they calculate the attention weights using the feature values projected using these matrices.\n ViT vs. Convolutional Neural Networks\nThe ViT model represents an input image as a series of image patches, like the series of word embeddings used when using transformers to text, and directly predicts class labels for the image.\n Transformer Architecture modified for images (ViT)\n\u200d\nThe paper suggests using a Transformer Encoder as a base model to extract features from the image and passing these “processed” features into a Multilayer Perceptron (MLP) head model for classification.\n Although transformer-based models are known to only be effective when large training datasets are available, we show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets\nAutonomous driving\nOn Tesla AI Day in 2021, Tesla revealed many intricate inner workings of the neural network powering Tesla FSD.\nThere has been a dramatic increase in the complexity of the computer vision model landscape. Many models are now at your fingertips, from the first ConvNets to the latest Vision Transformers. Just as supervised learning on ImageNet gave way to self-supervised learning and image-text pair training, such as CLIP, so did training paradigms. Compared to ResNets, CLIP's visual encoder is far more ...")],
  'generation': 'A Vision Transformer (ViT) distinguishes itself from '
                'traditional Transformer models by applying the transformer '
                'architecture to image recognition tasks, where it treats an '
                'input image as a series of image patches similar to how text '
                'transformers treat a series of word embeddings. Unlike '
                'traditional models that might rely on convolutional layers, '
                'ViTs use self-attention mechanisms to calculate attention '
                'weights for each pixel in relation to all other pixels, '
                'enabling it to directly predict class labels for the image. '
                'This approach allows for the encoding of the relative '
                "location of image patches to reconstruct the image's "
                "structure, leveraging the transformer's strengths in handling "
                'sequential data for computer vision tasks.',
  'question': 'What distinguishes a Vision Transformer from traditional '
              'Transformer models in machine learning?'}
'------------------------'
*****  Generated Answer  *****
('A Vision Transformer (ViT) distinguishes itself from traditional Transformer '
 'models by applying the transformer architecture to image recognition tasks, '
 'where it treats an input image as a series of image patches similar to how '
 'text transformers treat a series of word embeddings. Unlike traditional '
 'models that might rely on convolutional layers, ViTs use self-attention '
 'mechanisms to calculate attention weights for each pixel in relation to all '
 'other pixels, enabling it to directly predict class labels for the image. '
 'This approach allows for the encoding of the relative location of image '
 "patches to reconstruct the image's structure, leveraging the transformer's "
 'strengths in handling sequential data for computer vision tasks.')