{"text_format":"1","workbook":"xlsx","n_sheets":10,"sheets":["Intro","data","basic SGD","momentum","adagrad","rmsprop","adam","eve","adagrad_ann","adam_ann"],"not_rendered":{"charts":1,"images":0,"macros":1,"external_links":0,"defined_names":87,"comments":2,"pivot_tables":0},"unopened":[],"bounds":{"document_bytes":16777216,"sheet_cells":100000,"rows_first":2000,"rows_last":500,"rows_between":2000,"halvings":0,"cell_chars":16384,"cell_head_chars":4096},"cells_cut":0,"cells_raw":0,"read":"whole","date_system":"1900"}

{"sheet":"Intro","index":0,"n_rows":35,"n_cols":2,"rows_kept":35,"cut":false,"hidden":false,"merged":[]}
["What is this excel notebook about?"]
["This notebook explores different flavors/variations of Stochastic Gradient Descent, the key optimization method used in Deep Learning."]
["To keep things simple, instead of working with neural networks here, we will use a simple linear model (y=ax+b).  However, the optimization methods are the same for NNs."]
[]
["Goal: We want to create a model so that we can predict y when we're given x. "]
["To build the model, we are given training data x, y and we want to find an optimal a, b to create a model of the form y = ax + b."]
[]
["The key idea of SGD:"]
["1. Randomly choose some weights to start."]
["2. Use those weights to calculate a prediciton, calculate the error/loss, and then update those weights."]
["3. Repeat step 2 lots of times.  Eventually we end up with some decent weights."]
[]
["How does this relate to NNs?  To make an analogy with CNNs: x would be our images, y would be the dog/cat labels, and a and b are the weights.  "]
["We are trying to find the optimal weights (a, b). We use the same optimization methods to find weights for our NNs."]
[]
["Overview of the sheets in this notebook:"]
["data:","Here we randomly generate some training data x and y that we will use on the other sheets.  In the real world, you would get your x and y data from another source. In our CNN, x is the image data and y is the category (dog or cat)."]
[null,"We create our data by arbitrarily choosing a and b (a=2, b=30), randomly generating x, and then calculating y = ax + b.  We are going to pretend we don't know the true values of a and b, and see if our optimization algorithms help us find them."]
[null,"We can get a sense of how good our answers are, since really we know what the \"true\" a and b are."]
["basic SGD:","This is the most basic, vanilla form of stochastic gradient descent.  If we were doing gradient descent (not stochastic!), we would evaluate our error on all our data, and then update the weights a, b."]
[null,"What makes it stochastic is that we're just evaluating the error on a subset of our data (in this case, a single data point. For NNs, this is called our \"mini-batch\") and then updating the weights."]
["momentum:","Accelerate SGD in the relevant direction and dampen side-to-side oscillations"]
[null,"It does this by combining a decaying average of previous gradients with the current gradient"]
["adagrad:","Adapts the learning rate by dividing by the sqrt of the avg of the previous gradients squared"]
["rmsprop:","Adapts the learning rate by dividing by exponentially decaying average of squared gradients"]
["adam:","rmsprop + momentum"]
["eve:","Modifies learning rate based on recent volatility of loss"]
["adagrad_ann:","adagrad with learning rate annealling"]
["adam_ann: ","adam with learning rate annealling"]
[]
[null,"On each sheet, click 'reset' to return the a and b guesses to 1."]
[null,"Click 'run' to copy the last row's updated a and b guesses to the top row, and recalculate, a few times. (Each is effectively one epoch)"]
[]
["Some comments have been added to the other sheets in this notebook.  "]
["Comments are indicated by a small red triangle in the corner of a cell.  Hover your mouse over that cell to read the comment."]

{"sheet":"data","index":1,"n_rows":34,"n_cols":3,"rows_kept":34,"cut":false,"hidden":false,"merged":[]}
["weights:"]
["b","const",30]
["a","slope",2]
[]
[null,"x","y=a*x + b"]
[null,{"$f":"RANDBETWEEN(1,100)","v":76},{"$f":"$C$2+B6*$C$3","v":182}]
[null,71,{"$f":"$C$2+B7*$C$3","v":172}]
[null,79,188]
[null,25,80]
[null,51,132]
[null,6,42]
[null,6,42]
[null,65,160]
[null,20,70]
[null,8,46]
[null,11,52]
[null,24,78]
[null,69,168]
[null,31,92]
[null,9,48]
[null,63,156]
[null,29,88]
[null,40,110]
[null,27,84]
[null,68,166]
[null,45,120]
[null,96,222]
[null,82,194]
[null,94,218]
[null,69,168]
[null,14,58]
[null,86,202]
[null,45,120]
[null,26,82]

{"sheet":"basic SGD","index":2,"n_rows":37,"n_cols":14,"rows_kept":37,"cut":false,"hidden":false,"merged":[]}
[null,"intercept",1,null,"learn",1E-4,null,null,null,null,null,"de/da=x*2(ax+b-y)"]
[null,"slope",1,null,null,null,null,"e=(ax+b-y)^2",null,null,"de/db=2(ax+b-y)"]
["x","y","intercept","slope","y_pred","err^2","errb1","est de/db","erra1","est de/da","de/db","de/da","new a","new b"]
[14,58,{"$f":"C1","v":1},{"$f":"C2","v":1},{"$f":"C4+D4*A4","v":15},{"$f":"(B4-E4)^2","v":1849},{"$f":"(((C4+0.01)+A4*D4)-B4)^2","v":1848.1401000000001},{"$f":"(G4-F4)/0.01","v":-85.989999999992506},{"$f":"(((D4+0.01)*A4+C4)-B4)^2","v":1836.9795999999999},{"$f":"(I4-F4)/0.01","v":-1202.0400000000109},{"$f":"2*(E4-B4)","v":-86},{"$f":"K4*A4","v":-1204},{"$f":"D4-L4*$F$1","v":1.1204000000000001},{"$f":"C4-K4*$F$1","v":1.0085999999999999}]
[86,202,{"$f":"N4","v":1.0085999999999999},{"$f":"M4","v":1.1204000000000001},{"$f":"C5+D5*A5","v":97.363},{"$f":"(B5-E5)^2","v":10948.901769},{"$f":"(((C5+0.01)+A5*D5)-B5)^2","v":10946.809128999999},{"$f":"(G5-F5)/0.01","v":-209.26400000007561},{"$f":"(((D5+0.01)*A5+C5)-B5)^2","v":10769.665728999997},{"$f":"(I5-F5)/0.01","v":-17923.604000000341},{"$f":"2*(E5-B5)","v":-209.274},{"$f":"K5*A5","v":-17997.563999999998},{"$f":"D5-L5*$F$1","v":2.9201563999999998},{"$f":"C5-K5*$F$1","v":1.0295273999999999}]
[28,86,{"$f":"N5","v":1.0295273999999999},{"$f":"M5","v":2.9201563999999998},{"$f":"C6+D6*A6","v":82.7939066},{"$f":"(B6-E6)^2","v":10.279034889523562},{"$f":"(((C6+0.01)+A6*D6)-B6)^2","v":10.215013021523621},{"$f":"(G6-F6)/0.01","v":-6.4021867999940696},{"$f":"(((D6+0.01)*A6+C6)-B6)^2","v":8.5620225855236392},{"$f":"(I6-F6)/0.01","v":-171.70123039999226},-6.4121868000000006,{"$f":"K6*A6","v":-179.54123040000002},2.9381105230399998,1.0301686186799999]
[51,132,{"$f":"N6","v":1.0301686186799999},{"$f":"M6","v":2.9381105230399998},{"$f":"C7+D7*A7","v":150.87380529371998},356.22052626525243,356.59810237112754,37.757610587510726,375.73190766484646,1951.1381399594029,37.747610587439965,1925.1281399594382,2.745597709044056,1.0263938576212559]
[28,86,1.0263938576212559,2.745597709044056,77.903129710854827,65.559308479241835,65.397471073459073,-16.183740578276229,61.103461117320741,-445.5847361921094,-16.193740578290345,-453.42473619212967,2.7909401826632689,1.0280132316790849]
[29,88,1.0280132316790849,2.7909401826632689,81.965278528913885,36.417863233587767,36.297268804165981,-12.059442942178578,33.001824780357914,-341.60384532298519,-12.06944294217223,-350.01384532299471,2.8259415671955685,1.0292201759733022]
[72,174,1.0292201759733022,2.8259415671955685,204.49701301405423,930.06780277939322,930.67784303967369,61.004026028047065,974.50190151963125,4443.4098740238023,60.994026028108465,4391.5698740238095,2.3867845797931877,1.0231207733704912]
[62,154,1.0231207733704912,2.3867845797931877,149.00376472054813,24.962366967639472,24.862542262050525,-9.9824705588947182,19.151435221119364,-581.09317465201082,-9.992470558903733,-619.53317465203145,2.4487378972583906,1.0241200204263816]
[84,198,1.0241200204263816,2.4487378972583906,206.7181033901312,76.005326721017212,76.179788788819678,17.446206780246598,91.357340416437154,1535.2013695419942,17.43620678026241,1464.6413695420424,2.3022737603041863,1.0223763997483555]
[15,60,1.0223763997483555,2.3022737603041863,35.556482804311145,597.48553289593679,596.99676255202269,-48.877034391409779,590.17497773723017,-731.05551587066202,-48.887034391377711,-733.30551587066566,2.3756043118912529,1.0272651031874933]
[42,114,1.0272651031874933,2.3756043118912529,100.80264620262011,174.17014725321741,173.90630017726966,-26.384707594775136,163.26077006341862,-1090.9377189798788,-26.39470759475978,-1108.5777189799107,2.4864620837892439,1.0299045739469692]
[62,154,{"$f":"N14","v":1.0299045739469692},{"$f":"M14","v":2.4864620837892439},155.19055376888008,1.4174182765945609,1.4413293519721406,2.3911075377579749,3.2781049500058757,186.06866734113149,2.3811075377601583,147.62866734112981,2.4716992170551308,1.0296664631931931]
[47,124,1.0296664631931931,2.4716992170551308,117.19952966478435,46.246396780148054,46.110487373443668,-13.590940670438556,40.074854665045535,-617.1542115102518,-13.600940670431299,-639.24421151027104,2.5356236382061579,1.0310265572602362]
[35,100,1.0310265572602362,2.5356236382061579,89.777853894475754,104.4922710026845,104.28792808057392,-20.434292211058391,97.459268728817648,-703.30022738668561,-20.444292211048491,-715.5502273866972,2.6071786609448275,1.033070986481341]
[9,48,1.033070986481341,2.6071786609448275,24.497678934984791,552.35909544305764,551.88914902175748,-46.994642130016473,548.13677765135503,-422.23177917026078,-47.004642130030419,-423.0417791702738,2.649482838861855,1.0377714506943441]
[38,106,1.0377714506943441,2.649482838861855,101.71811932744484,18.334502094001436,18.248964480550413,-8.5537613451023731,15.224672782859551,-310.98293111418849,-8.5637613451103221,-325.42293111419224,2.6820251319732744,1.0386278268288551]
[44,118,1.0386278268288551,2.6820251319732744,119.04773363365292,1.0977457670875612,1.1188004397606006,2.1054672673039398,2.2133513647021283,111.56055976145672,2.0954672673058496,92.200559761457384,2.6728050759971285,1.0384182801021244]
[99,228,1.0384182801021244,2.6728050759971285,265.64612080381784,1417.2304115756463,1417.9834339917218,75.302241607550968,1492.7498307672063,7551.9419191560019,75.292241607635674,7453.9319191559316,1.9274118840815353,1.0308890559413608]
[13,56,1.0308890559413608,1.9274118840815353,26.087243549001322,894.77299849676308,894.17484336774316,-59.815512901991497,887.01258181950345,-776.04166772596272,-59.825512901997357,-777.73166772596562,2.005185050854132,1.0368716072315605]
[21,72,1.0368716072315605,2.005185050854132,43.145757675168333,832.56730014010714,831.99031529361059,-57.698484649654347,820.49261836367816,-1207.4681776428974,-57.708484649663333,-1211.87817764293,2.1263728686184251,1.0426424556965268]
[28,86,1.0426424556965268,2.1263728686184251,60.581082777012433,646.12135278909398,645.61307444463432,-50.827834445965436,631.96515914422116,-1415.6193644872815,-50.837834445975133,-1423.4593644873037,2.2687188050671554,1.0477262391411244]
[20,70,1.0477262391411244,2.2687188050671554,46.422102340484237,555.91725804259886,555.44580008940864,-47.145795319022454,546.52609897879279,-939.11590638060716,-47.155795319031526,-943.11590638063058,2.3630303957052186,1.0524418186730276]
[8,46,1.0524418186730276,2.3630303957052186,19.956684984314776,678.25425700621543,677.73349070590189,-52.076630031353943,674.09372660370593,-416.05304025094938,-52.086630031370447,-416.69304025096358,2.4046996997303149,1.0576504816761645]
[64,158,1.0576504816761645,2.4046996997303149,154.95843126441633,9.251140373280025,9.1904089985684063,-6.0731374711618713,5.7675323917329973,-348.36079815470276,-6.0831374711673334,-389.32079815470934,2.443631779545786,1.0582587954232812]
[99,228,1.0582587954232812,2.443631779545786,242.97780497045611,224.33464173301962,224.63429783242847,29.965609940884974,254.9707955745221,3063.6153841502478,29.955609940912211,2965.605384150309,2.1470712411307553,1.0552632344291899]
[70,170,1.0552632344291899,2.1470712411307553,151.35025011358206,347.8131708259458,347.44027582821781,-37.289499772799672,322.19352098496114,-2561.9649840984662,-37.299499772835873,-2610.9649840985112,2.4081677395406063,1.0589931844064735]
[27,84,1.0589931844064735,2.4081677395406063,66.079522152002838,321.14352630055703,320.78521674359689,-35.830955696013689,311.53936826263867,-960.41580379183529,-35.840955695994325,-967.70580379184673,2.5049383199197908,1.0625772799760729]
[17,64,1.0625772799760729,2.5049383199197908,43.646528718612515,414.26379320226511,413.85682377663744,-40.69694256276648,407.37251296659326,-689.1280235671843,-40.706942562774969,-692.01802356717451,2.5741401222765083,1.0666479742323505]
[8,46,1.0666479742323505,2.5741401222765083,21.659768952444416,592.44684744838878,591.96014282743761,-48.670462095117273,588.55881048077993,-388.80369676088549,-48.680462095111167,-389.44369676088934,2.6130844919525971,1.0715160204418617]
[null,null,null,null,"rmse",{"$f":"SQRT(SUM(F4:F32))","v":150.75521153771854},null,null,null,null,{"$f":"SQRT(SUMSQ(K4:K32))","v":301.51042307543707},{"$f":"SQRT(SUMSQ(L4:L32))","v":20791.831056415613}]
[]
[]
[null,null,"Paste error here:"]
[null,null,"rmse"]

{"sheet":"momentum","index":3,"n_rows":33,"n_cols":11,"rows_kept":33,"cut":false,"hidden":false,"merged":[]}
[null,"b",1.2610777567972657,"learn",null,null,null,null,"beta",0.9,{"$f":"1-J1","v":9.9999999999999978E-2}]
[null,"a",2.3580347909544339,1E-4]
["x","y","b","a","pred","de/db","de/da","new b","new a",{"$f":"J33","v":-19.279531290451075},{"$f":"K33","v":162.8396203477341}]
[14,58,{"$f":"C1","v":1.2610777567972657},{"$f":"C2","v":2.3580347909544339},{"$f":"C4+D4*A4","v":34.273564830159337},{"$f":"2*(D4*A4+C4-B4)","v":-47.452870339681326},{"$f":"F4*A4","v":-664.34018475553853},{"$f":"C4-J4*$D$2","v":1.2632874433168031},{"$f":"D4-K4*$D$2","v":2.3500226269706932},{"$f":"J3*$J$1+$K$1*F4","v":-22.0968651953741},{"$f":"K3*$J$1+$K$1*G4","v":80.12163983740686}]
[86,202,{"$f":"H4","v":1.2632874433168031},{"$f":"I4","v":2.3500226269706932},{"$f":"C5+D5*A5","v":203.36523336279643},2.7304667255928621,234.82013840098614,{"$f":"C5-J5*$D$2","v":1.2652488565171309},{"$f":"D5-K5*$D$2","v":2.3404634780013169},{"$f":"J4*$J$1+$K$1*F5","v":-19.614132003277405},{"$f":"K4*$J$1+$K$1*G5","v":95.5914896937648}]
[28,86,{"$f":"H5","v":1.2652488565171309},{"$f":"I5","v":2.3404634780013169},{"$f":"C6+D6*A6","v":66.798226240554001},-38.403547518891997,-1075.299330528976,1.2673981638726148,2.3426132372341679,-21.493073554838862,-21.497592328509256]
[51,132,{"$f":"H6","v":1.2673981638726148},2.3426132372341679,{"$f":"C7+D7*A7","v":120.74067326281518},-22.518653474369643,-1148.4513271928517,1.269557727027294,2.3560325338156622,-21.595631546791942,-134.19296581494348]
[28,86,1.269557727027294,2.3560325338156622,67.238468673865839,-37.523062652268322,-1050.645754263513,1.2718765644930279,2.3786163582816422,-23.18837465733958,-225.83824465980041]
[29,88,1.2718765644930279,2.3786163582816422,70.25175095466065,-35.496498090678699,-1029.3984446296822,1.2743184831930952,2.4092357847473211,-24.419187000673492,-306.19426465678856]
[72,174,1.2743184831930952,2.4092357847473211,174.73929498500021,1.4785899700004279,106.45847784003081,1.2765014241234558,2.4357286837880316,-21.829409303606102,-264.92899040710665]
[62,154,1.2765014241234558,2.4357286837880316,152.29167981898144,-3.4166403620371284,-211.83170244630196,1.2785002373644008,2.4616906099491342,-19.988132409449207,-259.61926161102622]
[84,198,1.2785002373644008,2.4616906099491342,208.06051147309168,20.121022946183359,1690.1659274794022,1.2800979590517894,2.4681546842193325,-15.977216873885952,-64.640742701983442]
[15,60,1.2800979590517894,2.4681546842193325,38.302418222341771,-43.395163555316458,-650.92745332974687,1.2819698602059924,2.4804816255958086,-18.719011542029001,-123.26941376475978]
[42,114,1.2819698602059924,2.4804816255958086,105.46219813522995,-17.075603729540092,-717.17535664068384,1.2838253272820703,2.4987476264010438,-18.554670760780112,-182.66000805235217]
[62,154,1.2838253272820703,{"$f":"I14","v":2.4987476264010438},156.20617816414679,4.4123563282935834,273.56609235420217,1.2854511240872577,2.5124513662022134,-16.257968051872744,-137.03739801169672]
[47,124,1.2854511240872577,2.5124513662022134,119.3706653355913,-9.2586693288174047,-435.15745845441802,1.2870069279052143,2.5291363066078105,-15.558038179567211,-166.84940405596885]
[35,100,1.2870069279052143,2.5291363066078105,89.806777659178579,-20.386444681642843,-713.52556385749949,1.2886110157881918,2.5512880086114227,-16.040878829774776,-221.51702003612189]
[9,48,1.2886110157881918,2.5512880086114227,24.250203093290995,-47.499593813418009,-427.49634432076209,1.2905296908210058,2.5754995038578814,-19.186750328139098,-242.11495246458591]
[38,106,1.2905296908210058,2.5754995038578814,99.159510837420498,-13.680978325159003,-519.87717635604213,1.2923933081337899,2.6024886213432548,-18.63617312784109,-269.89117485373151]
[44,118,1.2923933081337899,2.6024886213432548,115.80189264723701,-4.3962147055259777,-193.43344704314302,1.294114525862351,2.6287131615505221,-17.212177285609581,-262.24540207267268]
[99,228,1.294114525862351,2.6287131615505221,261.53671751936406,67.073435038728121,6640.2700688340838,1.2949928874676686,2.5859125470487219,{"$f":"J20*$J$1+$K$1*F21","v":-8.7836160531758125},428.00614501800283]
[13,56,1.2949928874676686,2.5859125470487219,34.911855999101057,-42.176288001797886,-548.29174402337253,1.2962051757924724,2.5528749114373355,-12.122883248038018,330.37635611386531]
[21,72,1.2962051757924724,2.5528749114373355,54.906578315976518,-34.186843368046965,-717.92371072898629,1.2976381037184763,2.5303202764943773,-14.329279260038913,225.54634942958018]
[28,86,1.2976381037184763,2.5303202764943773,72.146605845561027,-27.706788308877947,-775.79007264858251,1.2992048067349686,2.5177790057722009,-15.667030164922815,125.41270722176392]
[20,70,1.2992048067349686,2.5177790057722009,51.65478492217899,-36.690430155642019,-733.80860311284039,1.3009817437513682,2.5138299481533708,-17.769370163994736,39.490576188303507]
[8,46,1.3009817437513682,2.5138299481533708,21.411621328978335,-49.176757342043331,-393.41405873634665,1.3030727546395482,2.5142099368837871,-20.910108881799594,-3.7998873041615013]
[64,158,1.3030727546395482,2.5142099368837871,162.21250871520192,8.425017430403841,539.20111554584582,1.3048704142646061,2.5091599155857032,-17.976596250579252,50.500212980839223]
[99,228,1.3048704142646061,2.5091599155857032,249.71170205724923,43.423404114498453,4298.9170073353471,1.3060540738860134,2.4616257263440744,-11.836596214071484,475.34189241628991]
[70,170,1.3060540738860134,2.4616257263440744,173.61985491797122,7.239709835942449,506.77968851597143,1.3070469704469203,2.4137771591414485,-9.9289656090700902,478.48567202625804]
[27,84,1.3070469704469203,2.4137771591414485,66.479030267266026,-35.041939465467948,-946.13236556763457,1.3082909967463914,2.3801747723147617,-12.440262994709876,336.02386826686882]
[17,64,1.3082909967463914,2.3801747723147617,41.771262126097341,-44.457475747805319,-755.77708771269045,1.3098551951733932,2.3574903950478703,-15.641984270019421,226.84377266891289]
[8,46,1.3098551951733932,2.3574903950478703,20.169778355556357,-51.660443288887286,-413.28354631109829,1.3117795781905839,2.3412072909707793,-19.243830171906204,162.83104077091178]
[null,null,null,null,null,null,null,null,null,-19.279531290451075,162.8396203477341]

{"sheet":"adagrad","index":4,"n_rows":33,"n_cols":9,"rows_kept":33,"cut":false,"hidden":false,"merged":[]}
[null,"b",1,"learn",null,28.092710228958623,1918.710769527765]
[null,"a",1,1E-3,null,{"$f":"$D$2/F1","v":3.5596423123646386E-5},{"$f":"$D$2/G1","v":5.2118329447127722E-7}]
["x","y","b","a","pred","de/db","de/da","new b","new a"]
[14,58,{"$f":"C1","v":1},{"$f":"C2","v":1},{"$f":"C4+D4*A4","v":15},{"$f":"2*(D4*A4+C4-B4)","v":-86},{"$f":"F4*A4","v":-1204},{"$f":"C4-F4*F$2","v":1.0030612923886335},{"$f":"D4-G4*G$2","v":1.0006275046865434}]
[86,202,{"$f":"H4","v":1.0030612923886335},{"$f":"I4","v":1.0006275046865434},{"$f":"C5+D5*A5","v":87.057026695431361},-229.88594660913728,-19770.191408385806,1.0112444098143123,1.0109313981770935]
[28,86,{"$f":"H5","v":1.0112444098143123},{"$f":"I5","v":1.0109313981770935},{"$f":"C6+D6*A6","v":29.317323558772934},-113.36535288245413,-3174.2298807087159,{"$f":"C6-F6*F$2","v":1.0152798108830776},1.0125857537637304]
[51,132,{"$f":"H6","v":1.0152798108830776},{"$f":"I6","v":1.0125857537637304},{"$f":"C7+D7*A7","v":52.657153252833332},-158.68569349433335,-8092.9703682110012,1.0209284539723711,1.016803674722293]
[28,86,1.0209284539723711,1.016803674722293,29.491431346196574,-113.01713730760684,-3164.4798446129917,1.0249514598121958,1.0184529487529963]
[29,88,1.0249514598121958,1.0184529487529963,30.560086973649089,-114.87982605270182,-3331.5149555283529,1.0290407707087388,1.0201892786930988]
[72,174,1.0290407707087388,1.0201892786930988,74.48266883661185,-199.0346623267763,-14330.495687527893,1.0361256927651947,1.0276580936469311]
[62,154,1.0361256927651947,1.0276580936469311,64.750927498874915,-178.49814500225017,-11066.88499013951,1.0424795882614808,1.0334259692256267]
[84,198,1.0424795882614808,1.0334259692256267,87.850261003214129,-220.29947799357174,-18505.156151460025,1.0503214616940584,1.0430705474733499]
[15,60,1.0503214616940584,1.0430705474733499,16.696379673794308,-86.607240652411377,-1299.1086097861707,1.053404369677893,1.0437476211784742]
[42,114,1.053404369677893,1.0437476211784742,44.890804459173808,-138.21839108165238,-5805.1724254294004,1.0583244500103051,1.0467731800681332]
[62,154,1.0583244500103051,{"$f":"I14","v":1.0467731800681332},65.958261614234559,-176.08347677153088,-10917.175559834915,1.0645923919545472,1.0524630295927293]
[47,124,1.0645923919545472,1.0524630295927293,50.530354782812829,-146.93929043437436,-6906.1466504155951,1.0698229051103376,1.0560623978560946]
[35,100,1.0698229051103376,1.0560623978560946,38.032006830073648,-123.9359863398527,-4337.7595218948445,1.0742345829203375,1.0583231656543399]
[9,48,1.0742345829203375,1.0583231656543399,10.599143073809397,-74.801713852381198,-673.21542467143081,1.0768972563770007,1.0586740342872591]
[38,106,1.0768972563770007,1.0586740342872591,41.306510559292853,-129.38697888141428,-4916.7051974937422,1.0815029700239538,1.0612365389000329]
[44,118,1.0815029700239538,1.0612365389000329,47.775910681625405,-140.44817863674919,-6179.719860016964,1.086502422817653,1.0644573056555862]
[99,228,1.086502422817653,1.0644573056555862,106.46777568272068,-243.06444863455863,-24063.380414821306,1.0951546477775644,1.0769987375362984]
[13,56,1.0951546477775644,1.0769987375362984,15.096138235749443,-81.807723528501114,-1063.5004058705144,1.0980667101190673,1.0775530161815015]
[21,72,1.0980667101190673,1.0775530161815015,23.726680049930597,-96.546639900138814,-2027.4794379029152,1.1015034251641189,1.0786097045944205]
[28,86,1.1015034251641189,1.0786097045944205,31.302575153807894,-109.39484969238421,-3063.0557913867578,1.1053974905213166,1.0802061181029248]
[20,70,1.1053974905213166,1.0802061181029248,22.709519852579813,-94.580960294840366,-1891.6192058968072,1.1087642344034125,1.0811919984325391]
[8,46,1.1087642344034125,1.0811919984325391,9.7583002218637258,-72.483399556272545,-579.86719645018036,1.111344384163458,1.0814942155283409]
[64,158,1.111344384163458,1.0814942155283409,70.326974177977277,-175.34605164404545,-11222.147305218909,1.1175860764108403,1.0873430112319169]
[99,228,1.1175860764108403,1.0873430112319169,108.76454418837062,-238.47091162325876,-23608.620250702617,1.1260747878836634,1.0996474297120995]
[70,170,1.1260747878836634,1.0996474297120995,78.101394867730633,-183.79721026453873,-12865.804718517711,1.1326173111491857,1.1063528722013207]
[27,84,1.1326173111491857,1.1063528722013207,31.004144860584844,-105.99171027883031,-2861.7761775284184,1.1363902369158698,1.1078443821375643]
[17,64,1.1363902369158698,1.1078443821375643,19.969744733254462,-88.060510533491083,-1497.0286790693485,1.1395248761093042,1.1086246084764397]
[8,46,1.1395248761093042,1.1086246084764397,10.008521743920822,-71.982956512158353,-575.86365209726682,1.142087211887002,1.108924738991806]
[null,null,null,null,null,{"$f":"SQRT(SUMSQ(F4:F32))/COUNT(F4:F32)","v":27.319962416863692},{"$f":"SQRT(SUMSQ(G4:G32))/COUNT(G4:G32)","v":1857.066741115646}]

{"sheet":"rmsprop","index":5,"n_rows":33,"n_cols":11,"rows_kept":33,"cut":false,"hidden":false,"merged":[]}
[null,"b",1,"learn",null,null,null,null,null,0.9,{"$f":"1-J1","v":9.9999999999999978E-2}]
[null,"a",1,0.02]
["x","y","b","a","pred","de/db","de/da","new b","new a",{"$f":"J33","v":9309.6314749795092},{"$f":"K33","v":38847505.049484089}]
[14,58,{"$f":"C1","v":1},{"$f":"C2","v":1},{"$f":"C4+D4*A4","v":15},{"$f":"2*(D4*A4+C4-B4)","v":-86},{"$f":"F4*A4","v":-1204},{"$f":"C4-F4*$D$2/SQRT(J3)","v":1.0178263406803389},{"$f":"D4-G4*$D$2/SQRT(K3)","v":1.0038634466250291},{"$f":"J3*$J$1+$K$1*F4^2","v":9118.268327481559},{"$f":"K3*$J$1+$K$1*G4^2","v":35107716.144535683}]
[86,202,{"$f":"H4","v":1.0178263406803389},{"$f":"I4","v":1.0038634466250291},{"$f":"C5+D5*A5","v":87.350082750432847},-229.29983449913431,-19719.78576692555,1.0658524663353617,1.0704261418714132,13464.282904866439,70483939.599426061]
[28,86,{"$f":"H5","v":1.0658524663353617},{"$f":"I5","v":1.0704261418714132},{"$f":"C6+D6*A6","v":31.037784438734928},-109.92443112253014,-3077.884071430844,1.0847991240043706,1.0777583902566392,{"$f":"J5*$J$1+$K$1*F6^2","v":13326.192670140983},{"$f":"K5*$J$1+$K$1*G6^2","v":64382882.675200231}]
[51,132,{"$f":"H6","v":1.0847991240043706},1.0777583902566392,{"$f":"C7+D7*A7","v":56.050477027092967},-151.89904594581407,-7746.8513432365171,1.1111158584281933,1.0970678449350288,14300.905419051738,63945964.981100745]
[28,86,1.1111158584281933,1.0970678449350288,31.829015516609001,{"$f":"2*(D8*A8+C8-B8)","v":-108.34196896678199},-3033.5751310698956,1.1292353059910241,1.1046549863400486,14044.613101106479,58471626.290575251]
[29,88,1.1292353059910241,1.1046549863400486,33.16422990985243,-109.67154018029514,-3180.4746652285589,1.1477437132995656,1.1129735692503928,13842.93646354764,53636005.571133792]
[72,174,1.1477437132995656,1.1129735692503928,81.281840699327844,-185.43631860134431,-13351.41493929679,1.179265489384306,1.149434611195548,15897.305642834803,66098433.10214816]
[62,154,1.179265489384306,1.149434611195548,72.444211383508289,-163.11157723298342,-10112.917788444973,1.2051388606764128,1.1743123492204013,16968.113741294474,69715700.411518008]
[84,198,1.2051388606764128,1.1743123492204013,99.847376195190122,-196.30524760961976,-16489.64079920806,1.2352789907045649,1.2138104208910743,19124.877391072438,89934955.739056915]
[15,60,1.2352789907045649,1.2138104208910743,19.442435304070678,-81.115129391858645,-1216.7269408778798,1.2470099230776819,1.2163764339212277,17870.356073590989,81089502.610017031]
[42,114,1.2470099230776819,1.2163764339212277,52.33482014776925,-123.3303597044615,-5179.8751075873834,1.2654614958665862,1.2278809131934156,17604.358228715075,75663662.962035671]
[62,154,1.2654614958665862,{"$f":"I14","v":1.2278809131934156},77.394078113858356,-153.21184377228329,-9499.1343138815646,1.2885561911232954,1.2497218037710511,18191.309313053822,77120651.937148318]
[47,124,1.2885561911232954,1.2497218037710511,60.0254809683627,-127.9490380632746,-6013.6047889739066,1.3075291489616883,1.2634173555013009,18009.274015880168,73024930.999230474]
[35,100,1.3075291489616883,1.2634173555013009,45.527136591507222,-108.94572681698556,-3813.1004385944943,1.3237656368615158,1.2723416247149248,17395.263753460276,67176411.394788384]
[9,48,1.3237656368615158,1.2723416247149248,12.774840259295839,-70.450319481408314,-634.05287533267483,1.3344487407943169,1.2738888269071464,16152.062129617498,60498972.560181305]
[38,106,1.3344487407943169,1.2738888269071464,49.742224163265881,-112.51555167346824,-4275.5909635917933,1.3521550710901007,1.2848827361283015,15802.830853494237,56277143.112957954]
[44,118,1.3521550710901007,1.2848827361283015,57.886995460735371,-120.22600907852926,-5289.9443994552876,1.3712826930924653,1.29898585224858,15667.977094039874,53447779.976594999]
[99,228,1.3712826930924653,1.29898585224858,129.97088206570189,-196.05823586859623,-19409.765350991027,1.4026089584361303,1.3520847425934954,17945.062569826499,85776901.076988667]
[13,56,1.4026089584361303,1.3520847425934954,18.979710612151571,-74.040578775696858,-962.52752408405911,1.4136631582911605,1.3541632806325754,16698.757043387868,77291856.892751753]
[21,72,1.4136631582911605,1.3541632806325754,29.851092051575243,-84.297815896849514,-1770.2541338338399,1.4267099655061728,1.3581904393135271,15739.493515546994,69876051.173312157]
[28,86,1.4267099655061728,1.3581904393135271,39.456042266284932,-93.087915467430136,-2606.461633088044,1.4415497612853454,1.3644265979611134,15032.080164599438,63567810.280456945]
[20,70,1.4415497612853454,1.3644265979611134,28.730081720507613,-82.539836558984774,-1650.7967311796956,1.4550140699354706,1.3685675954784378,14210.154610057887,57483542.237178609]
[8,46,1.4550140699354706,1.3685675954784378,12.403554833762973,-67.19289033247405,-537.5431226597924,1.4662874439809293,1.3699855801769021,13240.627600175287,51764083.274332635]
[64,158,1.4662874439809293,1.3699855801769021,89.145364575302665,-137.70927084939467,-8813.3933343612589,1.4902227447383232,1.3944851740299169,13812.949167944951,54355265.153515711]
[99,228,1.4902227447383232,1.3944851740299169,139.54425497370011,-176.91149005259979,-17514.23751520738,1.5203280357227018,1.4419968577618538,15561.421782413567,79594590.212073892]
[70,170,1.5203280357227018,1.4419968577618538,102.46010807905246,-135.07978384189508,-9455.5848689326558,1.5419849070187279,1.4631939659832729,15829.934404449519,80575939.712225318]
[27,84,1.5419849070187279,1.4631939659832729,41.0482219885671,-85.9035560228658,-2319.3960126173765,1.5556402175293456,1.4683617245108218,14984.88305774193,73056305.527337328]
[17,64,1.5556402175293456,1.4683617245108218,26.517789534213314,-74.964420931573372,-1274.3951558367473,1.5678880294102748,1.471343708138098,14048.361192528348,65913083.275925606]
[8,46,1.5678880294102748,1.471343708138098,13.338637694515059,-65.322724610969885,-522.58179688775908,1.5789105635894394,1.4726310635401902,13070.230908335576,59349084.121776894]
[null,null,null,null,null,null,null,null,null,9309.6314749795092,38847505.049484089]

{"sheet":"adam","index":6,"n_rows":33,"n_cols":13,"rows_kept":33,"cut":false,"hidden":false,"merged":[]}
[null,"b",1,"learn",null,null,null,null,"beta",0.7,{"$f":"1-J1","v":0.30000000000000004},0.9,{"$f":"1-L1","v":9.9999999999999978E-2}]
[null,"a",1,1]
["x","y","b","a","pred","de/db","de/da","new b","new a",{"$f":"J33","v":-19.370509429769257},{"$f":"K33","v":-636.96204313606268},{"$f":"L33","v":5489.999378391848},{"$f":"M33","v":27555653.260482278}]
[14,58,{"$f":"C1","v":1},{"$f":"C2","v":1},{"$f":"C4+D4*A4","v":15},{"$f":"2*(D4*A4+C4-B4)","v":-86},{"$f":"F4*A4","v":-1204},{"$f":"C4-J4*$D$2/SQRT(L4)","v":1.5222168817019814},{"$f":"D4-K4*$D$2/SQRT(M4)","v":1.1615923752588633},{"$f":"J3*$J$1+$K$1*F4","v":-39.359356600838481},{"$f":"K3*$J$1+$K$1*G4","v":-807.07343019524387},{"$f":"L3*$L$1+$M$1*F4^2","v":5680.5994405526626},{"$f":"M3*$L$1+$M$1*G4^2","v":24945049.53443405}]
[86,202,{"$f":"H4","v":1.5222168817019814},{"$f":"I4","v":1.1615923752588633},{"$f":"C5+D5*A5","v":101.41916115396423},-201.16167769207155,-17299.904281518153,{"$f":"C5-J5*$D$2/SQRT(L5)","v":2.4406800624791023},{"$f":"D5-K5*$D$2/SQRT(M5)","v":1.956762413138724},{"$f":"J4*$J$1+$K$1*F5","v":-87.90005292820841},{"$f":"K4*$J$1+$K$1*G5","v":-5754.9226855921179},9159.1415536862842,52379213.395959646]
[28,86,{"$f":"H5","v":2.4406800624791023},{"$f":"I5","v":1.956762413138724},{"$f":"C6+D6*A6","v":57.230027630363374},-57.539944739273253,-1611.118452699651,3.2915880452759305,2.6120846873989083,-78.792020471527863,-4511.7814157243774,8574.3119223775175,47400862.323226608]
[51,132,{"$f":"H6","v":3.2915880452759305},2.6120846873989083,{"$f":"C7+D7*A7","v":136.50790710262027},9.015814205240531,459.80652446726708,3.8883391853633813,3.0743899544554814,-52.44967006849734,-3020.3050336668839,7725.0092207181069,42681918.294898212]
[28,86,3.8883391853633813,3.0743899544554814,89.971257910116861,7.9425158202337229,222.39044296654424,4.299897696724396,3.4047229603450924,-34.332014301878019,-2047.4963906768555,6958.8166544017631,38418672.216320679]
[29,88,4.299897696724396,3.4047229603450924,103.03686354673208,30.073727093464157,872.13808571046059,4.4882135114777197,3.603749969127918,-15.010291883275361,-1171.6060477606607,6353.3778950908027,34652867.478743285]
[72,174,4.4882135114777197,3.603749969127918,263.95821128868783,179.91642257737567,12953.982425571048,4.0288745185189248,3.161053900045363,43.467722454919958,3066.0704942388525,8955.0320168858034,47968146.799069315]
[62,154,4.0288745185189248,3.161053900045363,200.01421632133145,92.028432642662892,5705.7628238450998,3.4139176986042741,2.5948473334506703,58.03593551124284,3857.9781931207267,8906.4520566637366,46426905.05935967]
[84,198,3.4139176986042741,2.5948473334506703,221.38109370846058,46.762187416921165,3928.0237430213779,2.8116321550862153,2.0055441344621592,54.653811082946341,3878.9918580909225,8234.4770681988884,43327151.605997667]
[15,60,2.8116321550862153,2.0055441344621592,32.894794172018607,-54.210411655962787,-813.15617483944175,2.5610608767426699,1.6101187170542504,21.9945442612736,2471.3474482118131,7704.9062345698949,39060558.741865851]
[42,114,2.5610608767426699,1.6101187170542504,70.186046993021193,-87.627906013957613,-3680.37205258622,2.6851705330629283,1.5065431264185785,-10.89219082129577,625.83159797240296,7702.2806023520043,36509016.712225035]
[62,154,2.6851705330629283,{"$f":"I14","v":1.5065431264185785},96.090844371014796,-115.81831125797041,-7180.7352979941652,3.1509878847706978,1.7848847743732295,-42.370026952298169,-1716.1384708175678,8273.4406643816146,38014410.982988462]
[47,124,3.1509878847706978,1.7848847743732295,87.040572280312489,-73.918855439375022,-3474.1862056506261,3.7307891539045204,2.1618594860757017,-51.834675498421227,-2243.5527912674852,7992.496316890175,35419966.863842919]
[35,100,3.7307891539045204,2.1618594860757017,79.395871166554087,-41.208257666891825,-1442.2890183412139,4.2977129677247019,2.5154990768569427,-48.646750148962411,-2003.1736593896037,7363.0587351952527,32085989.938701395]
[9,48,4.2977129677247019,2.5154990768569427,26.937204659437189,-42.125590681125622,-379.13031613013061,4.8637420958165807,2.7975327047649166,-46.690402308611368,-1515.9606564117616,6804.209400699101,28891764.924492151]
[38,106,4.8637420958165807,2.7975327047649166,111.1699848768834,10.339969753766809,392.91885064313874,5.2414257968623277,2.9824643287675414,-29.581290689897912,-943.29680429529139,6134.4799580800718,26018026.954362009]
[44,118,5.2414257968623277,2.9824643287675414,136.46985626263415,36.939712525268305,1625.3473511118054,5.3693900077862287,3.0179543917582352,-9.6249897253480459,-172.70355767316221,5657.4861984170111,23680399.660102423]
[99,228,5.3693900077862287,3.0179543917582352,304.14687479185153,152.29374958370306,15077.081208786603,4.91693659294081,2.354626035348419,38.950632067367295,4402.2318722647688,7411.0761948016752,44044197.47172679]
[13,56,4.91693659294081,2.354626035348419,35.527075052470252,-40.945849895059496,-532.29604863577345,4.7357574924828016,1.890708652583319,14.981687478639255,2921.8734959946059,6837.6248376843823,39668111.632893443]
[21,72,4.7357574924828016,1.890708652583319,44.440639196732505,-55.11872160653499,-1157.4931537372347,4.8110245818103294,1.6070483714152493,-6.0484352469130211,1698.0635010750534,6457.6697010698153,35835279.509698957]
[28,86,4.8110245818103294,1.6070483714152493,49.808378981437308,-72.383242037125385,-2026.7307770395107,5.1370238348986579,1.5054536647111667,-25.948877283976731,580.62521764068401,6335.8361037433415,32662515.322988983]
[20,70,5.1370238348986579,1.5054536647111667,35.246097129121992,-69.507805741756016,-1390.1561148351202,5.6331196071810004,1.5074040165516844,-39.016555821310519,-10.609182102057332,6185.3859992723765,29589517.193051435]
[8,46,5.6331196071810004,1.5074040165516844,17.692351739594475,-56.61529652081105,-452.9223721664884,6.210424645925567,1.5351626641089862,-44.296178031160679,-143.30313912138666,5887.3765793590746,26651079.341267183]
[64,158,6.210424645925567,1.5351626641089862,104.46083514890068,-107.07832970219863,-6853.0131009407123,6.9967870169029807,1.9377730023851494,-63.130823532472064,-2156.2161276671845,6445.2157906044431,28682350.263306968]
[99,228,6.9967870169029807,1.9377730023851494,198.83631425303275,-58.327371493934493,-5774.4097778995147,7.7840089368128922,2.5382014930761638,-61.689787920910788,-3241.6742227368836,6140.9024380831434,29148496.065286424]
[70,170,7.7840089368128922,2.5382014930761638,185.45811345214435,30.916226904288692,2164.1358833002087,8.2362201674663407,2.8516922305650976,-33.907983473350939,-1619.9311909257556,5622.3935028745755,26701994.870896537]
[27,84,8.2362201674663407,2.8516922305650976,85.231910392723975,2.4638207854479504,66.52316120709466,8.5594807333501723,3.0789329154024836,-22.996442195711268,-1113.9948852859006,5060.761193873399,24032237.916904584]
[17,64,8.5594807333501723,3.0789329154024836,60.901340295192391,-6.1973194096152184,-105.35442996345871,8.8254395857997245,3.2533973748521992,-17.956705359882452,-811.40274868916799,4558.5257512725384,21630124.080805417]
[8,46,8.8254395857997245,3.2533973748521992,34.852618584617318,-22.294762830765364,-178.35810264612292,9.1242979833935127,3.39424432669359,-19.258122601147328,-621.48935487625442,4152.3788211132924,19470292.83400283]
[null,null,null,null,null,null,null,null,null,-19.370509429769257,-636.96204313606268,5489.999378391848,27555653.260482278]

{"sheet":"eve","index":7,"n_rows":38,"n_cols":14,"rows_kept":38,"cut":false,"hidden":false,"merged":[]}
[null,"b",1,"learn",null,null,null,null,null,"beta",0.7,{"$f":"1-K1","v":0.30000000000000004},0.9,{"$f":"1-M1","v":9.9999999999999978E-2}]
[null,"a",1,1]
["x","y","b","a","pred","err^2","de/db","de/da","new b","new a",{"$f":"K33","v":4.223800054218521},{"$f":"L33","v":-15.966661111217437},{"$f":"M33","v":672.90108904624503},{"$f":"N33","v":2666772.9314021971}]
[14,58,{"$f":"C1","v":1},{"$f":"C2","v":1},{"$f":"C4+D4*A4","v":15},{"$f":"(E4-B4)^2","v":1849},{"$f":"2*(D4*A4+C4-B4)","v":-86},{"$f":"G4*A4","v":-1204},{"$f":"C4-K4*$D$2/SQRT(M4)/$F$38","v":1.6228226547538451},{"$f":"D4-L4*$D$2/SQRT(N4)/$F$38","v":1.2334176425301189},{"$f":"K3*$K$1+$L$1*G4","v":-22.84333996204704},{"$f":"L3*$K$1+$L$1*H4","v":-372.37666277785223},{"$f":"M3*$M$1+$N$1*G4^2","v":1345.2109801416204},2545057.2382619777]
[86,202,{"$f":"I4","v":1.6228226547538451},{"$f":"J4","v":1.2334176425301189},{"$f":"C5+D5*A5","v":107.69673991234407},{"$f":"(E5-B5)^2","v":8893.1048631600788},-188.60652017531186,-16220.160735076819,{"$f":"C5-K5*$D$2/SQRT(M5)/$F$38","v":2.6738301975603509},{"$f":"D5-L5*$D$2/SQRT(N5)/$F$38","v":2.1920598705892598},{"$f":"K4*$K$1+$L$1*G5","v":-72.572294026026498},-5126.7118844675433,4767.9318273914887,28599912.941608552]
[28,86,{"$f":"I5","v":2.6738301975603509},{"$f":"J5","v":2.1920598705892598},{"$f":"C6+D6*A6","v":64.05150657405963},481.73636366854765,-43.89698685188074,-1229.1156318526607,3.6291518100984779,2.9698084175435473,-63.969701873782768,-3957.4330086830787,4483.8331901197589,25890994.171094153]
[51,132,{"$f":"I6","v":3.6291518100984779},2.9698084175435473,{"$f":"C7+D7*A7","v":155.08938110481938},533.11951980359026,46.178762209638762,2355.1168726915766,4.1035948476136941,3.3923174511015532,-30.925162648756306,-2063.668044270682,4248.6976790292192,23856552.302388392]
[28,86,4.1035948476136941,3.3923174511015532,99.088483478457178,171.3083997658465,26.176966956914356,732.95507479360197,4.3247009343852767,3.6562878801418925,-13.794523767055107,-1224.6811085513966,3892.3512710326359,21524619.386316121]
[29,88,4.3247009343852767,3.6562878801418925,110.35704945850016,499.83766048982216,44.714098917000314,1296.7088685930091,4.2629442172318788,3.7622194071224295,3.7580630381615219,-468.26411540807476,3703.0512081253009,19540302.836673282]
[72,174,4.2629442172318788,3.7622194071224295,275.14274153004681,10229.854164213855,202.28548306009361,14564.55478032674,3.5281320209872931,3.1133733788703055,63.316289044741154,4041.5815533123709,7424.6877529983121,38798898.147919804]
[62,154,3.5281320209872931,3.1133733788703055,196.55728151094624,1811.1222096019267,85.114563021892479,5277.1029073573336,2.7164407849577321,2.3948077767174509,69.855771237886557,4412.2379595258599,7406.667861539252,37703789.842611745]
[84,198,2.7164407849577321,2.3948077767174509,203.88029402922362,34.577857870122955,11.76058805844724,987.88939690956818,2.0749742244971272,1.8145609824457738,52.427216284054765,3384.9333907409723,6679.8322185333755,34031003.40440321]
[15,60,2.0749742244971272,1.8145609824457738,29.293388961183734,942.89596148915302,-61.413222077632533,-921.19833116448797,1.8463392290631262,1.436876399369766,18.275084775548574,2093.093874169334,6389.0073812756991,30712763.700496912]
[42,114,1.8463392290631262,1.436876399369766,62.195148002593299,2683.7426904732129,-103.6097039948134,-4351.6075677821627,2.0677583700487943,1.4074937876967266,-18.290351855560022,159.68344158388459,6823.6037193374141,29535136.172845121]
[62,154,2.0677583700487943,{"$f":"J14","v":1.4074937876967266},89.332373207245837,4181.9019550069361,-129.33525358550833,-8018.7857223015162,2.6515327799079595,1.8067322323520627,-51.603822374544514,-2293.8573075817362,7814.0041294064467,33011715.001579277]
[47,124,2.6515327799079595,1.8067322323520627,87.567947700454894,1327.2944347567898,-72.864104599090211,-3424.6129161572399,3.3182328069249198,2.2805404350114982,-57.981907041908229,-2633.0839901543877,7563.5214903685173,30883340.863972452]
[35,100,3.3182328069249198,2.2805404350114982,83.13714803232736,284.35577648364102,-33.72570393534528,-1180.3996377370847,3.9277269397414845,2.6962748014300804,-50.705046109939346,-2197.2786844291968,6920.9116519251229,27934341.10805219]
[9,48,3.9277269397414845,2.6962748014300804,28.194200152612204,392.26970759478644,-39.611599694775592,-356.50439725298031,4.5206009984374624,3.0242778904283361,-47.377012185390214,-1645.0463982763317,6385.7283697705252,25153616.535773043]
[38,106,4.5206009984374624,3.0242778904283361,119.44316083471423,180.71857322799471,26.886321669428469,1021.6802234382818,4.8496030568660711,3.2014726576531061,-25.098012028944602,-845.02841176194761,5819.4429620846704,22742637.930092227]
[44,118,4.8496030568660711,3.2014726576531061,145.71439999360274,768.08796700540756,55.42879998720548,2438.8671994370411,4.8622263571214077,3.1709374434965127,-0.93996842409957537,140.14027159774912,5544.7338526783669,21063181.458731994]
[99,228,4.8622263571214077,3.1709374434965127,318.78503326327615,8241.922264614157,181.5700665265523,17975.436586128679,4.2710894099923173,2.4040974824814256,{"$f":"K20*$K$1+$L$1*G21","v":53.813042061095992},5490.7291659570283,8287.0293732561931,51268495.359052129]
[13,56,4.2710894099923173,2.4040974824814256,35.524356682250854,419.25196927568527,-40.951286635498292,-532.36672626147777,3.9804152044489416,1.8619515382709513,25.383743452117706,3683.8003982914761,7626.0272236408482,46169987.256269954]
[21,72,3.9804152044489416,1.8619515382709513,43.081397508138913,836.28557008227426,-57.837204983722174,-1214.5813046581657,3.9754946994893468,1.5190550696342595,0.41745892136573559,2214.2858874065832,7197.9387293096734,41700509.305205472]
[28,86,3.9754946994893468,1.5190550696342595,46.509036649248614,1559.536186370389,-78.981926701502772,-2211.4939476420777,4.2531912629467268,1.3752742956170223,-23.402356765494822,886.55193689198472,7101.9593309268621,38019528.922730677]
[20,70,4.2531912629467268,1.3752742956170223,31.758677175287175,1462.3987713839019,-76.482645649425649,-1529.652912988513,4.7240156336475074,1.347726933970729,-39.326443430674075,161.69048192783526,6976.7229063877367,34451559.833879031]
[8,46,4.7240156336475074,1.347726933970729,15.505831105413339,929.89433657157667,-60.988337789173322,-487.90670231338657,5.2859153829545518,1.3536848933958632,-45.82501173822385,-33.18867334453131,6651.0083503775941,31030209.145507362]
[64,158,5.2859153829545518,1.3536848933958632,91.921748560289799,4366.3353133295632,-132.1565028794204,-8458.0161842829057,6.101574990746788,1.7860116067024805,-71.724459080582818,-2560.636926626044,7732.4416406716591,35080992.008315779]
[99,228,6.101574990746788,1.7860116067024805,182.91672405429236,2032.5017699968212,-90.166551891415281,-8926.4886372501132,6.9779023997436784,2.4969323831555772,-77.25708692383256,-4470.3924398132649,7772.1981846032213,39541112.746579736]
[70,170,6.9779023997436784,2.4969323831555772,181.76316922063407,138.37215011327274,23.526338441268138,1646.8436908887697,7.5379139801433137,2.9370034672785499,-47.022059314302346,-2635.2216006026542,7050.3272261882084,35858210.886143781]
[27,84,7.5379139801433137,2.9370034672785499,86.837007596664165,8.0486121035301821,5.67401519332833,153.19841021986491,7.9296583561727489,3.2536146989050541,-31.213236962013141,-1798.6955973558984,6348.5139484107995,32274736.772818793]
[17,64,7.9296583561727489,3.2536146989050541,63.24110823755867,0.57591670710130782,-1.5177835248826597,-25.802319923005214,8.2247306336053132,3.4886669499086054,-22.304600930873995,-1266.8276141260303,5713.8929202525596,29047329.671508256]
[8,46,8.2247306336053132,3.4886669499086054,36.134066232874154,97.33664909731398,-19.731867534251691,-157.85494027401353,8.5238709554993228,3.6713570289290445,-21.532780911887304,-934.13581197042515,5181.43828786623,26145088.52257432]
[null,null,null,null,"loss",{"$f":"SQRT(SUM(F4:F32))","v":235.28150716589968},null,null,null,null,4.223800054218521,-15.966661111217437,672.90108904624503,2666772.9314021971]
[null,null,null,null,"ft-1",239.79849927509557]
[null,null,null,null,"ft-2",119.89924963754778]
[null,null,null,null,"r_t",{"$f":"ABS(ft_1-ft_2)/MIN(ft_1,ft_2)","v":1}]
[null,null,null,null,"dt-1",1,null,"beta"]
[null,null,null,null,"dt",{"$f":"dt_1*H38+(1-H38)*rtee","v":1},null,0.9]

{"sheet":"adagrad_ann","index":8,"n_rows":33,"n_cols":10,"rows_kept":33,"cut":false,"hidden":false,"merged":[]}
[null,"b",29.406245790980623,"learn",null,0.27585466085056881,16.186122587302258,100,null,8.1502795969492121E-3]
[null,"a",2.0217110111321754,3.90625E-3,null,{"$f":"$D$2/F1","v":1.4160536522948314E-2},{"$f":"$D$2/G1","v":2.4133327663441694E-4}]
["x","y","b","a","pred","de/db","de/da","new a","new b"]
[14,58,{"$f":"C1","v":29.406245790980623},{"$f":"C2","v":2.0217110111321754},{"$f":"C4+D4*A4","v":57.710199946831082},{"$f":"2*(D4*A4+C4-B4)","v":-0.57960010633783554},{"$f":"F4*A4","v":-8.1144014887296976},{"$f":"D4-G4*$G$2","v":2.0236692862313777},{"$f":"C4-F4*$F$2","v":29.414453239455124}]
[86,202,{"$f":"I4","v":29.414453239455124},{"$f":"H4","v":2.0236692862313777},{"$f":"C5+D5*A5","v":203.45001185535361},2.9000237107072167,249.40203912082063,1.9634802749310449,29.373387347782238]
[28,86,{"$f":"I5","v":29.373387347782238},{"$f":"H5","v":1.9634802749310449},{"$f":"C6+D6*A6","v":84.350835045851497},-3.2983299082970063,-92.353237432316178,1.9857681843283821,29.420093468913411]
[51,132,{"$f":"I6","v":29.420093468913411},{"$f":"H6","v":1.9857681843283821},{"$f":"C7+D7*A7","v":130.6942708696609},-2.6114582606782051,-133.18437129458846,2.0179100050494001,29.4570731189919]
[28,86,29.4570731189919,2.0179100050494001,85.958553260375112,-8.2893479249776192E-2,-2.3210174189937334,2.0184701437882513,29.45824693513233]
[29,88,29.45824693513233,2.0184701437882513,87.993881104991615,-1.2237790016769168E-2,-0.35489591048630587,2.0185557919811932,29.458420228804822]
[72,174,29.458420228804822,2.0185557919811932,174.79443725145072,1.5888745029014331,114.39896420890318,1.9909475151050753,29.435920913376105]
[62,154,29.435920913376105,1.9909475151050753,152.87466684989076,-2.2506663002184837,-139.54131061354599,2.0246234768213034,29.467791555721316]
[84,198,29.467791555721316,2.0246234768213034,199.53616360871078,3.0723272174215595,258.07548626341099,1.9623412741023341,29.42428575394857]
[15,60,29.42428575394857,1.9623412741023341,58.859404865483583,-2.2811902690328338,-34.217854035492508,1.9705991809361176,29.456588632069003]
[42,114,29.456588632069003,1.9705991809361176,112.22175423138594,-3.5564915372281121,-149.37264456358071,2.0066477706881947,29.506950460375478]
[62,154,{"$f":"I14","v":29.506950460375478},{"$f":"H14","v":2.0066477706881947},153.91911224304354,-0.16177551391291445,-10.030081862600696,2.0090683632090074,29.509241288448759]
[47,124,29.509241288448759,2.0090683632090074,123.93545435927211,-0.12909128145577142,-6.0672902284212569,2.0105326022401244,29.511069290254607]
[35,100,29.511069290254607,2.0105326022401244,99.879710368658962,-0.24057926268207552,-8.4202741938726433,2.0125646946014921,29.514476021690481]
[9,48,29.514476021690481,2.0125646946014921,47.627558273103915,-0.7448834537921698,-6.7039510841295282,2.0141825810830221,29.525023971043243]
[38,106,29.525023971043243,2.0141825810830221,106.06396205219808,0.12792410439615765,4.8611159670539905,2.0130094320385932,29.523212497090775]
[44,118,29.523212497090775,2.0130094320385932,118.09562750678887,0.1912550135777451,8.4152205974207845,2.0109785592782163,29.520504223485808]
[99,228,29.520504223485808,2.0109785592782163,228.60738159202921,1.2147631840584268,120.26155522178425,1.9819554441033922,29.503302525051215]
[13,56,29.503302525051215,1.9819554441033922,55.268723298395315,-1.4625534032093697,-19.013194241721806,1.9865439605690336,29.524013065934124]
[21,72,29.524013065934124,1.9865439605690336,71.241436237883832,-1.5171275242323361,-31.859678008879058,1.9942327610554338,29.545496405650987]
[28,86,29.545496405650987,1.9942327610554338,85.384013715203139,-1.2319725695937223,-34.495231948624223,2.0025576084098593,29.562941798217988]
[20,70,29.562941798217988,2.0025576084098593,69.614093966415169,-0.77181206716966244,-15.436241343393249,2.0062828871121798,29.573871071183998]
[8,46,29.573871071183998,2.0062828871121798,45.62413416808144,-0.75173166383711987,-6.0138533106969589,2.0077342300368493,29.584515994865221]
[64,158,29.584515994865221,2.0077342300368493,158.07950671722358,0.15901343444716076,10.176859804618289,2.0052782151143518,29.582264279319091]
[99,228,29.582264279319091,2.0052782151143518,228.10480757563994,0.20961515127987695,20.751899976707818,2.0002700910965832,29.579296016313627]
[70,170,29.579296016313627,2.0002700910965832,169.59820239307444,-0.80359521385111066,-56.251664969577746,2.013845489719833,29.590675355689033]
[27,84,29.590675355689033,2.013845489719833,83.964503578124521,-7.0992843750957491E-2,-1.9168067812758522,2.0143080789810335,29.591680652445838]
[17,64,29.591680652445838,2.0143080789810335,63.834917995123405,-0.33016400975319016,-5.6127881658042327,2.0156626315401418,29.596355951964512]
[8,46,29.596355951964512,2.0156626315401418,45.721657004285646,-0.55668599142870789,-4.4534879314296631,2.0167374063750856,29.604238924277951]
[null,null,null,null,null,{"$f":"SQRT(SUMSQ(F4:F32))/COUNT(F4:F32)","v":0.29214010037172083},{"$f":"SQRT(SUMSQ(G4:G32))/COUNT(G4:G32)","v":16.630193282037062},{"$f":"SUMPRODUCT(histgrad,F2:G2)","v":8.1502795969492121E-3}]

{"sheet":"adam_ann","index":9,"n_rows":34,"n_cols":14,"rows_kept":34,"cut":false,"hidden":false,"merged":[]}
[null,"b",29.990939773126538,"learn",null,null,null,null,"beta",0.9,{"$f":"1-J1","v":9.9999999999999978E-2},0.95,{"$f":"1-L1","v":5.0000000000000044E-2}]
[null,"a",2.0041632844175137,3.90625E-3]
["x","y","b","a","pred","de/db","de/da","new b","new a",{"$f":"J33","v":-4.8441943141925917E-2},{"$f":"K33","v":-4.0642025509844446},{"$f":"L33","v":2.527589588662521E-2},{"$f":"M33","v":161.89615404770484},10.326280808548518]
[14,58,{"$f":"C1","v":29.990939773126538},{"$f":"C2","v":2.0041632844175137},{"$f":"C4+D4*A4","v":58.049225754971729},{"$f":"2*(D4*A4+C4-B4)","v":9.8451509943458859E-2},{"$f":"F4*A4","v":1.378321139208424},{"$f":"C4-J4*$D$2/SQRT(L4)","v":29.991782162981227},{"$f":"D4-K4*$D$2/SQRT(M4)","v":2.0052716484223598},{"$f":"J3*$J$1+$K$1*F4","v":-3.3752597833387438E-2},{"$f":"K3*$J$1+$K$1*G4","v":-3.5199501819651577},{"$f":"L3*$L$1+$M$1*F4^2","v":2.4496736082801299E-2},153.89633480345904]
[86,202,{"$f":"H4","v":29.991782162981227},{"$f":"I4","v":2.0052716484223598},{"$f":"C5+D5*A5","v":202.44514392730417},0.8902878546083457,76.56475549631773,{"$f":"C5-J5*$D$2/SQRT(L5)","v":29.992255287786474},{"$f":"D5-K5*$D$2/SQRT(M5)","v":2.0044351254225292},{"$f":"J4*$J$1+$K$1*Intro!F5","v":-3.0377338050048695E-2},{"$f":"K4*$J$1+$K$1*G5","v":4.4885203858631293},6.290252248181781E-2,439.30960727383217]
[28,86,{"$f":"H5","v":29.992255287786474},{"$f":"I5","v":2.0044351254225292},{"$f":"C6+D6*A6","v":86.116438799617299},{"$f":"2*(D6*A6+C6-B6)","v":0.23287759923459816},{"$f":"F6*A6","v":6.5205727785687486},{"$f":"C6-J6*$D$2/SQRT(L6)","v":29.992318613563189},2.0035402920398968,-4.0518443215840153E-3,4.6917256251336914,6.2468995168990425E-2,419.47002037817111]
[51,132,{"$f":"H6","v":29.992318613563189},2.0035402920398968,{"$f":"C7+D7*A7","v":132.17287350759793},0.3457470151958546,17.633097774988585,29.991845918855169,2.0023911752672543,3.0928041630159837E-2,5.9858628401191805,6.532259533638303E-2,414.04282621637793]
[28,86,29.991845918855169,2.0023911752672543,86.05879882633829,0.1175976526765794,3.2927342749442232,29.991228470377429,2.0012660249753647,3.9595002734801793E-2,5.7165499836016842,6.2747925965315948E-2,393.88278985582866]
[29,88,29.991228470377429,2.0012660249753647,88.02794319466301,5.5886389326019525E-2,1.6207052904545662,29.990569778911592,2.0001945431891333,4.1224141393923566E-2,5.3069655142869721,5.9766694092645116E-2,374.31998464496257]
[72,174,29.990569778911592,2.0001945431891333,174.00457688852919,9.1537770583727252E-3,0.65907194820283621,29.989946572899314,1.9991915359955668,3.8017104960368482E-2,4.8421761576785585,5.6782548969734579E-2,355.62570420435986]
[62,154,29.989946572899314,1.9991915359955668,153.93982180462444,-0.12035639075111249,-7.4620962265689741,29.989576019017193,1.9984271071706123,2.2179755389220387E-2,3.6117489192538059,5.4667704560979573E-2,340.62856299887062]
[84,198,29.989576019017193,1.9984271071706123,197.85745302134862,-0.2850939573027631,-23.947892413432101,29.989717116028224,1.9982489983603595,-8.5476158799779565E-3,0.8557847859852159,5.5998247557458081E-2,352.27221240119303]
[15,60,29.989717116028224,1.9982489983603595,59.963452091433616,-7.3095817132767138E-2,-1.0964372569915071,29.989970561806697,1.9981079610527384,-1.5002436005256872E-2,0.66056258168754367,5.346548510370052E-2,334.7187105140593]
[42,114,29.989970561806697,1.9981079610527384,113.91050492602172,-0.17899014795656853,-7.5175862141758785,29.990506438971764,1.9981422563139677,-3.140120720038804E-2,-0.15725229789879835,5.2394084501791206E-2,320.80848011273469]
[62,154,29.990506438971764,{"$f":"I14","v":1.9981422563139677},153.87532633043776,-0.24934733912448337,-15.459535025717969,29.991410045072325,1.9985126490828387,-5.319582039279757E-2,-1.687480570680715,5.2883085053124647E-2,316.71791726766799]
[47,124,29.991410045072325,1.9985126490828387,123.92150455196574,-0.15699089606852112,-7.3785721152204928,29.992504672528892,1.9990185421900744,-6.3575327960369923E-2,-2.2565897251346927,5.1471237872888279E-2,303.60418772726001]
[35,100,29.992504672528892,1.9990185421900744,99.958153649181497,-8.3692701637005484E-2,-2.929244557295192,29.993659147117956,1.9995526521259299,-6.5587065328033475E-2,-2.3238552083507429,4.9247899394608906E-2,288.85300202471916]
[9,48,29.993659147117956,1.9995526521259299,47.989633016251325,-2.0733967497349681E-2,-0.18660570747614713,29.994762357456022,2.0000502375806035,-6.1101755544965095E-2,-2.1101302582632835,4.6806999295287512E-2,274.4120930079863]
[38,106,29.994762357456022,2.0000502375806035,105.99667138551897,-6.6572289620694391E-3,-0.25297470055863869,29.99579334677934,2.0005158157515477,-5.565730288667553E-2,-1.9244147024928191,4.4468865265395806E-2,260.69468816754312]
[44,118,29.99579334677934,2.0005158157515477,118.01848923984744,3.6978479694880662E-2,1.6270531065747491,29.996674351630034,2.0009052312318709,-4.6393724628519914E-2,-1.5692679215860623,{"$f":"L19*$L$1+$M$1*F20^2","v":4.2313792400153247E-2},247.79231884974666]
[99,228,29.996674351630034,2.0009052312318709,228.08629224358526,0.17258448717052488,17.085864229881963,29.997143004447146,2.0008320428200688,{"$f":"J20*$J$1+$K$1*F21","v":-2.4495903448615443E-2},0.29624529356073981,4.1687373040741237E-2,249.99904073135733]
[13,56,29.997143004447146,2.0008320428200688,56.007959561108038,1.5919122216075721E-2,0.20694858880898437,29.997544437627624,2.0007592168388335,-2.0454400882146326E-2,0.2873156230855643,3.9615675311310689E-2,237.50123008070995]
[21,72,29.997544437627624,2.0007592168388335,72.013487991243124,2.6975982486248995E-2,0.56649563221122889,29.997860642614782,2.0006772416562133,-1.5711362545306796E-2,0.31523362399813071,3.7671276727300078E-2,225.64221444174015]
[28,86,29.997860642614782,2.0006772416562133,86.016823408988756,3.3646817977512455E-2,0.94211090337034875,29.998082968022022,2.0005764221574505,-1.0775544493024872E-2,0.37792135193535248,3.5844318308935669E-2,214.4044823673656]
[20,70,29.998082968022022,2.0005764221574505,70.009611411171036,1.9222822342072732E-2,0.38445644684145464,29.998247522693223,2.0004728065457664,-7.7757078095151127E-3,0.37857486142596269,3.4070578238428628E-2,203.6916485869732]
[8,46,29.998247522693223,2.0004728065457664,46.002029975059358,4.0599501187159603E-3,3.2479600949727683E-2,29.998390652309073,2.0003762178641762,-6.5921420166920059E-3,0.34396533537833918,3.236787348625552E-2,193.50711890384844]
[64,158,29.998390652309073,2.0003762178641762,158.02246859561635,4.4937191232691021E-2,2.8759802388922253,29.998422659906399,2.0002043647254926,-1.4392086917537046E-3,0.59716682572972779,3.0850447369736914E-2,184.24532607538094]
[99,228,29.998422659906399,2.0002043647254926,228.01865476773017,3.7309535460337884E-2,3.6936440105734505,29.998367150048402,1.9999371417264775,2.4356657234554536E-3,0.90681454421410002,2.937752507306338E-2,175.71521007545417]
[70,170,29.998367150048402,1.9999371417264775,169.99396707090185,-1.2065858196308454E-2,-0.8446100737415918,29.998344109304171,1.9997159526654782,9.8551333147906309E-4,0.73167208241853088,2.7915928066110884E-2,166.96511788051475]
[27,84,29.998344109304171,1.9997159526654782,83.990674831272088,-1.8650337455824229E-2,-0.50355911130725417,29.99836756242027,1.9995273369080135,-9.780717472512657E-4,0.60814896304595245,2.6537523417166142E-2,158.62954057541802]
[17,64,29.99836756242027,1.9995273369080135,63.990332289856504,-1.9335420286992644E-2,-0.32870214487887495,29.99843676166153,1.9993636351738933,-2.8138066012254031E-3,0.51446385225346969,2.5229340170191565E-2,150.70346580164951]
[8,46,29.99843676166153,1.9993636351738933,45.99334584305268,-1.3308313894640378E-2,-0.10646651115712302,29.998534219933632,1.9992159522331712,-3.8632573305669004E-3,0.45237081591241046,2.3976728722617899E-2,143.16885926746696]
[null,null,null,null,null,{"$f":"SQRT(SUMSQ(F4:F32))/COUNT(F4:F32)","v":3.8686302499625166E-2},{"$f":"SQRT(SUMSQ(G4:G32))/COUNT(G4:G32)","v":2.9941290342098443},null,null,-4.8441943141925917E-2,-4.0642025509844446,2.527589588662521E-2,161.89615404770484,{"$f":"SQRT(L33)+SQRT(M33)","v":12.882825901628111}]
[null,null,null,null,null,null,null,null,null,null,{"$f":"1/K3","v":-0.24605072888352394},{"$f":"1/L3","v":39.56338499278089},null,{"$f":"avg_grad2/prev_grad2","v":1.2475765612497367}]