From 07a699341035bcca92cb5ba3cb53d13dddddcf29 Mon Sep 17 00:00:00 2001 From: Irv Lustig Date: Thu, 18 Jul 2024 11:31:11 -0400 Subject: [PATCH 1/7] notebook and data from July 2024 webinar --- 2407webinar/mcfdata.v1.xlsx | Bin 0 -> 7033 bytes 2407webinar/mcfdata.v2.xlsx | Bin 0 -> 11679 bytes 2407webinar/netflow.ipynb | 384 ++++++++++++++++++++++++++++++++++++ 3 files changed, 384 insertions(+) create mode 100644 2407webinar/mcfdata.v1.xlsx create mode 100644 2407webinar/mcfdata.v2.xlsx create mode 100644 2407webinar/netflow.ipynb diff --git a/2407webinar/mcfdata.v1.xlsx b/2407webinar/mcfdata.v1.xlsx new file mode 100644 index 0000000000000000000000000000000000000000..ad8593d7bb14d4a31fb8492a5c7afe2a0b626b7e GIT binary patch literal 7033 zcmZ{J1z3~a|NiJ0A|W6nR9dJ z*s~ef*s!`-S|~&)09)Acp54f;Pg|+&KIC{foH_T+U51Z0%2>iF;L<~-IT9{B!~V98 zs7ter+LoWti#@KT-@L%IiTKZzJgBT|hG5~=8 zzvDHswuAm0a8Hb~d@DO%XJhEnmoJqgSQ*`6MoQRzDDBlzQ!xG5z@@C|YQADf&>F@r zhx5(lwzEw!?$Z;ePm*TN1PKN7Kdk&p2Caay1VmSPF&wjn{X-I;einx#h3tu@m+E z2eNmNlLGiPZrXY&^jDV>H)BA78RSU<3C?+QTkCsJ-Rd&Y=zY*ZsWDnAa>)dA|MfxH zgys7pS5L_9#m73h8j~`g-@(bZqo7%}@uQCIttY$pI5 z%xQ4a;ES`9=9ca@VP`+rt$Jw&Xyy0(Mo7>)ST{Ihlnsa?+6K|YW6yL9Jg(y zzwMfI;I;(KE$+GS!@388clcpOM}JlFI%}QS^r_V14>tm$s%5dE zhX73*r_t|&DDQM1pF75ZCMfEs#kF@ew*^A*uRGdNN6iLF-*HRXrADFQ0{Geno`XF~ z#bKdyEir@IQKNm2tH2=4q_4h5(teNHr%JDm66(KYak-^>3iiH^^EN9)%H66@?u)&& zUt3DgjU7{X+H3Qf8Efx?8e$McZzV)%CrEDc)T8hCI69k_vv21N`-h^i$piIwWjcL* zOKETIsOzVXH9e1&q^+pO@|!BC#^R)CB-6jL)GgujSIOStBz%II8klGfCP&V~;&_)4 zcv{&}gg-~1s7bv8Wsq|iD^L?c5}cGCY8PYSe}DSPrdWk_r{j*aGAE;`bz zf;!iMu4j!4)GSp><``r;DC~BPc9z0}O&(1~fowunA@#`LtbUiWlqMpnxCNi-si+|- zFUYQn*cWD-@2~%opku&Aaj+8qQmcq3R#gA%W>2(Z?`SkGevB?3;@RG?DEvmsKz_NT zt&g^vs8?yat+LH)e%-IGYPPNauPSk|RZ3G$~U zM%Q8wy@;}IJzH;g0WAnjpw@aOYFMt04-8zaHY=B9?g+o5=nUTf_-H#LFtM*yBFNQU zFVQ5v1I&HT(xQllS*lvUW^9l7-kh;M%Lj}%rOL?rFVyroARQ=7Q6D{?zm;6GSxwN^ zTO5d4gy%r3tV$m{lUh=}5NI(=Bs3}Pnt_%#t5Ni+LH5fU%tb1R6#K5Viod4g;42aX z31p1oNOI&p9+0XutTM^Asqr^Ta={1^?tflf>9kc;sUdKJ-9_fCtry65gL0VU?9Z~` zU@=32wFnjzbhbuTlT%9EsGQCdoSXIcuWY_a-(413)jQOYDDgU;UOcyo6_R=t)+V$Gf3j-506TKcPU8jp~4NrW_uaM?+WtI{5nOws`Z;0vlf^`Hhi zefd^5t5g>H8@TSUNF|Ag)IsL^Im(J4`((G87RjO;%~8rT&RGE)xZn(m?g}BZM3z^x zXK1JoexYGQU8&L;ZHiE(`=0N^mA}%WPvN{3r4l9uH#Bjov~Ix%DFN z85ZZ17k5l@;JEr|9^hoW;-3wYOagI@gcS*dV_S=lh%G;X58p98zQau(ji$@-9>$xw zq=+p%b-i<^tfehtek-gI>F)s2>)psYiUa^iAcDyK-vPwg+U~WzDHQ5p&-Ux%R}@K& zx3o_F8D+5-P=@`5!D>F0Sp_hX4n$C4<;#N+7L z1g1tcqgvlagH163kK6r zIGc7Roki~WqPW5A@Z)qzif>@4ahYPKHWC}7(H0U!)y8osOyDly}YZ=w{YgzaG~Rg%5LlPE!@W|b=8ZI z?EzW%Lm}NpJ4d`?TpCp=qG{Tkn+;FnFGBv)KIPI>7U&hcDwJ_|*e{5vbjXD`roE|e zfPZU2$PRbh36W;a;{2x;IDcjosR`2&2rWqIq3r=)W1i{YXVDE5d@&`mtSfd~Z=%rz zCW7eAW*c~u`SU&n1s9G78m)Z5@@OMrok=DabMx-{X#UQN6?*ktCUylpxDql3&Zov; zm17)HC6`qJe$WEOqmiOD$Hm2|c@b2ne$KU{rvRad9BS7>rQYa zxcSo9Dnyo{oT*y}B;i-ccgW>|I+&E24A#A|)A18lr-hvR zmSAGbWY72&q;nNzFdQltM9hvUc-kx6aq6s|)_dtnTakOG$aOCk%Bgp5KifSgrWD|@ zoX}@Ol$40fr#eVOe7S(dlGK(@Svh94<1d)3cFwz%1(tqam+~CuXOgSh@vK1ZkU+L{ z4jV%cm5pT>l-pnrtZ058$9FR#9hb#HK{|wQg{{^V+VLv%4Ox^Q7o6qg;+z~&FH`bv z9<2i~J(fNA+r$D9X{l)*2`-f`|EU7c4co{iZ$+LFuU%qz_=HOyDK6D3{u&fUC=zeE zt+d4kUh@6&Z(t7cDKSTMgM$_UXL?1!mqtOO!=?&B=JOvQz|}IT6w!ygT!s}HhWu;9 ztpZthO(;sg&hzl;9*D9P-cx*Z!%FVpJb&q%+tDJ!f-XCMk1F7e=3Q3Ge(X0F$%6N3 zM&CSoB|0npYW32|1b+L0TfYtqbu^Ep0IV-ch$qCkKDj|tt{+F5;ylguHp23 zCza0c*-nuZ2G-kUg-6-AxuBoF8r+o}l*uo^+3Q<=>xnbbWjVrcU84S{R=Iv@HBQ+&g&has zx4JM)*QN&)h#nTF;uCr`9(X(OD=-(r^Iy zaEj5AYZT9MECv3_SA86~DOMi~QegU8yV7K7BBm-n%-9gR$TWkUuay=!-n;bpI4dhK zHN;{T*7AAEr{$4R!X|l5LQtTG&#DCO^+qX#|x4@!V-zcoW0j)0$3Eopq@P zC-yWvJ$CnJ|8_}l6uyE+5M~zo@9@O^%gphXk;%Wzynv#;h5{8t8k`(5b1zqM9O2!9 zV&^;~F??6Eb-a~eW7$TNq~IB_wX}0dE=~6&(~FA*AlWFG{~W*i*%h@>j(Nu73X(h6 zDO-Hn_EU){`_SvTqlX(b$@Jes9kugnz6Xt~CkRl=VxvJQ-W!eCUb){P=#vxeh44Fj}Gsu zDBd;%7CueY311`TTjNwpn3g0_N1IFvm>V}etSLVa`0)lz7t^n)ydO0YHnx+wG5u&0 zBUFUIfG@E)b^F6Kdi~79!4cco!5g-&Cx1&}ZW}C}kBCmhKmg#*?@Yzs!PNq4|I_}{ z>hMrFJKh!1{N23R#H5-wUtTo@EE=vln5W!!hiQOBu7pEsh13miC- zm(9Af6-SvwD&eC?7S`xnUB>USeyCX=OUtWB45-Z#9pBsA5D%meE(Dz3&|1 zHY)=-1}(^ovr@ov>lW7Q$IL&L25Q()Qlu}AN}Kujj3rr=ygUo&JsHnaf^ofO95*ht zzP+)MPU+oNK(9Jqqpf`qQ#T0-gtP?(ds8YbR$VX@g*iP9g@h9Xc8p zWW963=~k?!vN|ELdAy5*luk3o`aT<{dj}Uj7FYBU;1r*B{5A#(MwC!O&A8qhhcsH( zG#}+!mdvj)OU$y0`t4ESH!lQj6_L_UWy7et_frbiUJ>@_ON_Ae?XgjYcikb4!)3hE zDPFf4BHX_w2a9-GKh29BjFm}^E8(Hqt9z8murQtD5GC&Dn?x}j`D5;RyMR`pPPSd9 z6?^zSw17G04ioOakd39&@ROiAS}pZ^wy5PU=vq>bT*%D)y8!kdqL>eMnuAW1-CUKj z$8P+`ZfD;Kq4nfV2rUyLbd8WTK+g_pVb8|;^PU=0Yt_n*Bj(-SpT>^S|0tLXx<`z9 zhNS-S?l=KjsAOtm>14KT6p++aM3`WT(oAfNKMNL?fBy>ah1tuI8YCGQZKPRm^wW}I zW=fu*L9u6WBZ;z?%%Wnfq_3*K7!&8(XK4g$kELddz}b| z8PUGAHLSem@%wgn(+lwT=1q&KH`GV?@lF%~;O_6u`?Jh2w6=czbNg1s4k#ifE9srx zh0$+?Lz5Fvg2#@9WN5mhweaG#tTrtxN&{uJnj~#jNP#rFea^IgGIWfXBiBFR#%uG6$!IzL!Gy8h0-tFrcrt`T^waNq~+%8=iE9BMRt zA%id*nZ~P<@1pGUxD@*u`R67Wj0gAo7x~6t5HZU0P*5@~@S%-b8Bqyn6ioDwKQ-p| zUQ)m~%W=-v(A?3j@`>B{gx88~v~Vr8QyQ>`va}DbdhWb-f1&tQP2@1|{TU&$_f+X` z2UgkBB!at7)9>dO^f$4*AC;JU)QMSwf|K7tm(Of^H=DC=Kq_WkD#p2Ji=7fUMpIPt z%&e_y3rk5xB7~mifSTrDC?*SLHpLx(K+U|G3oT#3+w_Op`a=hH`fK=!8FBQ}+vr1B zPj&zGE;2GyaHNKGEY4O|b<} z3CZj1$i975rrH%Tv4bSzYu6~0OHy>cHD!+IY$fiXl6;r$Iv`hGAwx@Gz1L$3{Jyb z?gkQtfcK;yEtrWVSN-E9SvHsHDTb`@&H9jT(aopfkElx-i&#Q3_p++Z@G? zBYn2JMF?4%)Dg69euf%936e`gG>^7)(xliE=^`ZzJG)ef{00K3BDzzV8kIkxh}%HZ z-&Eo9FZ!#d(KqZX#It@lY#+4?X~Gka z*F=fpF~my^j}4*6X+3$x*_c3zVz(lA?z5@s-)$=`U$?|J(4}J@7y9Uc}%3OGo@R z!R;Q%KLjw}yO_TS{SeyZUZcjhA2<~035^{58+i+ezwv8 F{|9ATVaNag literal 0 HcmV?d00001 diff --git a/2407webinar/mcfdata.v2.xlsx b/2407webinar/mcfdata.v2.xlsx new file mode 100644 index 0000000000000000000000000000000000000000..8c188e319554effecff8ea4df313374ddeeb7e84 GIT binary patch literal 11679 zcmeHNRa6{n)@_2jCRlKHLU8xs4#8c6ySqCyuEE`dy9IZb5Ht{iyTjkf%>8qRd*^YU zW@@djTHRe|ukNq*{*KB?f~f~PJEpI@u#`~)U8g6u)2}?;#J)b-$nO~<;U}>1-F`cRcA83R9! zpP1ivq9JsPWtaP=(}6A8*m%woFyQG5u0YwV1g_z8^ym1g>l))>Em4(Mie$a#TbIF5 zd;|7%&*9KjPa1$eHDOvM@PsY#W}3)VZDQwTD0C#PtNKe;Z4UG?GvyHeJP)faf+UB7 z7psJ(V)oXGcRz}Vw;>6w5o%v5g;zK0s8)B{Yj>{_E@xS0Cn zS!Q2^3}~Y27cg~W8>zE`DTr72PpT2z^eb=unV&ENqqn|m!So*l+I)FHf0yMB!_6zp zhanM;bh(n0YW zMG40r{GE53*Zt7)3U}m>0iw$d=CW`!3@(y-=klPWTU$qHYI3_oQQNZhUR0;qi`mPR zcM`6YPHoZDrHw_|l7p)xqSGhB)hOfi?{Q$!^Ir$O;z{vU?~_*hs()PpHY=!jP##p( zz?StRZamFAkT&jS{WA$24`zKbA4-T^Pg@kPesoj*;%MqF+;EFi0NrmV!O zekFlgIm*^gxij4|RjKshh>M&^I(3?AStK;_VEtxvnQ*4= zXiCNdT|s)#3$CC~YSq2c*j3M~;ES6wEa6uXh|e_E{6s4Vjupnjl^P?>W90%?`c-!ga-eBx z_(r80U`K$NIv8fkikbjbd{FdMT&BJ+PS4*vhj-Vo4(wV4gZ1T_K*IqBHj{BS;HU|k zqVdWI_u-ppPWCI4eD*Ub!}xK+@3Ez{+d;%972h5*3ztjL&ES?6i|G#Y=DLr4dWubl92%t!BWc-n zu4~HVbA%Ckj1ZODE!B}cSQX2&(3!ihr9>+_KHdeNC?mz&dYh8N&fW)(4iXFUZ2Fso zh<*>ztszV9HcHzL$+GXPqD$$lJt!|p;2>kkmMhQVO!He3#ZzNqZW-7^z@>=Z51fLo zmutJM|LU$WJL6~@f=ZqYjVf zF`4GC$NR)R+C4p-F}**^b~tLVKVi#3+aY}}7;c^Rl+}Cy`Jlgw&sU>y5FR8;JqQ2* z2LJ;G65l_Z)So5zhrEiN=^Kyvnxe`|?VW>Wesr~zzdWBO>5z?L!fXA;kJr;`waEgsB@9dl*kLu8fZkflQH` zk>}nlgt%s5LfM@JVJ|}rRv~wu>a^76I1aVEtR_zp;cvd(hawO|%OB7fLMCAhX4izh zcr&ofC~MyxGl?JWe3?D(0Ev@ds5=$N6(`e03g1Jj$e53d}9qwGBt5_o^Ch)FJknY*cs>L9O?dXLZ7t2}rw$iJW@mGkTk zX4-$fIB;oZNiFMw{$?TW%9jgK=8#Ym4s7q>xjdwVm0yrBHObE7f^pz%YR9oo*9n3m zL?d00nzu9bxR6xWYsdpYJZX&Y;)Wtb&H0 zu8Cb`i+qwL()%csyJVAv?Fu^v)1g_%vnv%I(lkYV$Vo{i`f410uQl0xrtKS>Ah5ij ztKYL{(ay zL424L=4KR9N-UFCYJrA9;|a~byFn@D`}OdCo#JOKqx=|AeAY{2@cEM!c$w=4!!U_x z2OsSs$%+~prdd+QR2LrK-U7Ktq-HKnUKbkhU5M&sO4Y&CcMHZ%shNmdfMJWt=^ChzE2PVP`HbQJLWh%l5Avy6^ zeeliGNV4{<093h*7v1a-ED*i$P$fweGkHFY_{Zy0_PQf_?CU8xsb1a4q{nC*c`k92-bnQbP%F^JWm8t59FiLMv6nc7w@5%pzyO{~N}SqvjYQ1!#?KJPe8}3Uy}w7pdbc_z z+QHpH*}_i3QrC<03-)?G}Y4X$xOje{iRJqN=3d5P}c!9XzF1{7p<;W9qv5=5@l*qybA<1>-vxUt%h zWa{#b0)GX4!xN@YW8uZ3E9PEwq-yT_9K7#am{)Ga$#e6ZyJoAS4eQHqlpbRcQ)1kw ziXT5Q-lw{%mN@uAD?N4quXV?VkgH0)yR9y(95mZii%RlV7j^d5N2X&gm@ww4R|-voQM3n zF0e8x%=Tme8g}^d%ce`xX7SlcDaFef{p`&&4Ickg7F6$4uoB5j%92qCKVcpHTd&Nb zj^Dh9Pdc#j<3|zE`FsrcysqI-s;nI#2YjrNepByxR}&t4_(d)8#^juxL+bEw*3E+; zNMm`Wx#02k`m1LKahTJli8Ef|`+nM`2f4Y6{)Sa2th;qLE|I$=(&*G?w*6AygZBl< zeY-2}inAwe#hZ@;>9Gqfy^yeWp|3)F;zB463f*&T*{b$rqz-5aIX=B3HMk628SYq& zvMZ5iwc%IeZ)75lFjXu~r`@xOFI2LS4eNC||7!aMh$QoG4B<|Sa#4072 z?#8gO;%{kump;xkf1}mD=T_<#%ZicP$=@FMX07dOWFGZ9$>2lu?0Z6r0si(Kzr1hn zt6Hp;g{FeS;<1kVII51C$V&6MwXBt;r-Cvuum&0!p;~uJMm(=VOK8l5)Zh@dG_b(G z+Us;b;4wV)KmLBs|9Sme*$Z{&gIr}9sCxHrT;ejps4}h$!CF-zJn#x4WP?c!#qE5oNOuVg|?EYDD(Bh@0t$YJs35# zUg?_$_!4$q`^^zCa~NpPWHlWvLm?Pb?=d4^|Ck$LLvmTPnhG-#x9ew43d)3OY6;RS zm4HjusGimBhTY&l@p-sYXML2K-oQ!Uw)$l0*8Kwtm8RH`yZY5%NLGs{((_%!9NIqHRKS=L=?rj6^8KYh_euWdVX|57rzDPP2zI zdcC8kUeYC7qXTmCL{)MDKWIo|9Dkd1Cr0pmI8NN0Tt0R3AQ(fu6KjtG26b>gE(O8Q z4b-g*59kAqc*oY**=66uTe-)&;F(EZ9hy{+TK{H6Ja8S!j}WC_=kp1GDC84|A&n=h zsD)<@dDM@C&7xgyyKOe84&08rRdMG!r~UJygx6Nl7N#&HDJ^?n0~NJm_6Gx4FTTyH znON|#lJXq`Db0L%z+Ovs`@=VAPI7Y4g_a=C_wCx@CxaiP+eUETG!)< z*$GsoDMX16lQITJgh_?WqYmBIhJV3uSRI1{GT7Doq!!lpEa z)TpXvfnFtQY*xB>G`iEl5!L=eIM2#yF8YC`^@R16<>A^h6A5)VleTg}%s~r{v!(|y zTt<4NJo_h1y(zA3U8+qmeECI3vit{d8m?})_uYrcRgGHoIs-LX9fK8)$0>dEyQFOk z8MS~MOZY? z7DBWp7(3YUEF}no?T2w0(onShEL+a6dy&gx9ULHnV^X&+&(aL2;mI{diyA1L+MT`7 z))kg-PP>S`1r=PLo_lqxQ(5d@^jJjrieAIw%s{7e!q8-rtB$wBzTcy*%HSe^|1j6tmGsjm`1BM5uNZ zUX{E3L8-VNt#Ccs(6|6;G@Pd?ejR_E%CeXoj<}H@2?b+9aTY!bojBs5#LU-t{Qcs_ zyPS~sctJd1j{3`m4JL9gG(J~oY%uQlcsKnVh205%1UJU$u*`9*u6Y#DdW6OaREb!t zM(3_vaOp;r-PrKG?lhoq(H^Xza6IAHMDYwL_+*_?nE>HGwSpMm zvL1SK#`g{S{}Ji;j*E$V}9pC zahIiC1nOhQf=ecDu-fV`NQAvE*7}fy>fA&~#nt?Bu$tdD$81v4_p2M_5|o&a_Ee#5 zT8^}Dt6=b!gu1$t1qIYCCY}Y|)#m-UO;EGjfxyrN5i0Ps?4=UJ`xnABG4GheHM%mq zj;7pv^19Umu*bz{l2NcVmpz`^N?nA=NXcL>P3*uM{Dg}=42(pAsv^D_WyT?VMQjqQ z_Irm~Dg1%}Y$d+DOoci!0b`n$A6v$E|6n~P2;zceP}g3fb$IQKq5?yJ)N9kJ-bKBt z$)XZeJ7O-WETIu~u;b8c`H$RkVEU@mE|YN7+vsH$+FD(>pUDdn_vsGyQe>+u7f>Qq z5M@!3jipN4)X#=4Big|j>q7RNS~9F0MaKa)Sbha6iAdKX&t@&T+=yc96X=KRsbm7K)W_xwqMl&ip z`WaWQ+muzE-#qxz4e|opa9}YCvmNR9LIUh#E+LNbWWE}LYeC{?^{^xRNpK%1rZUr@ z8Bz+N*Mb<%Hk>FJ=1?yFYg4!_7(;<%qws@r6Pk5?$o2@>Zwjw|++0;=@`-?69A4M^ z*Xy$V(fD|L_mfk0tcPo~^wu6X2PWQL4{vTaci9*Syx04OnI7i}cyGS#j6d)XWHkvh zw0oQ`f1+S$zdgS?!gpWOc!4~0%qtx#+%SN;v_tjX7-3UF25Q5D>lU_P3@II8iM(4O z@p`Qs)|gi_t+B6#Cd=j8XTJXmPYoIEjmk{VjVO?_Q)^5u4Zaf0%2b;S{;FSX--N!; z=nHROFz;8qbdHLHDEp$+=`06}Rq+s#ZTy~;?l*%Wg12}Cs2h`q`1w&<&SsR2TS9?{ z@-uxi=0ILNc3b;oicxjZ4*|hgvh+BZU{N91gKDD9Nq?N9fzhC!F{+>9=B+ zwnIZu9ihq_=%>2G-AolUFI_mvBXX^T^Ue>N==fKG)7H90@3k9_YQ%pK2j-9vjHV@@ zG1;X>bHd8lSB@}C4lN*W!3bjH@>F;nYaJSgctP>4SroS%DQq;}oeSCo_eF`F0Vb{c za*CE@9;D4KoApO1l{TF}ySQ;$c1>G6bXy7iB)NCN0zP3cvd^wQ?BmBcxoy#GKtx_DidXubKn>otuGPu$^t9R zkhGJUPkLyBTC(L}aW4zWKf%s7v86p$m32!8*4_Da%is+JDlMy$voepA5<{pO2icH3 zeZU)WlZRUn^9HAynooQ#EK7s$8 zW04;(YJv@y9<`sBPbssd<=GJ3n8;KDLlweUg+U@kBV_}pNMVY_I(f7SQ!)ijIV!Tl z*^pH{R|hruHwLi;JFKU1rO6#wP0TWpC2*oITW}E$?e#C{mAbRxzQH28zo2iDI`A>| z-BWcz6o9-M-0mO@zU>(0knG80k@EcTu}@5nJz9wvX3@XKFy{zEe|BTPhEz>pKv+p! zGex5sdMl=?y}}pKvmTn#SB`Bqj}jcpnd$M%4zWK}wUymtaYnTC_E|l2GglQ|;)-{l zcAg6LZGLXnF3+8mEDH^ECC9?J>&#MEUDnBmhmm)dNJ+e6Prhf-VKIvY6z9H6J9ly# zsP}Wa%QO)3o`bCTVwhqY_|OyViz2vEsqvPoVTYI9%ao0=^qec$Is;Q|yAI=K`)c;v zHTHHV7b`nA5n8GX)s9Wm7s)ulOT(}qm!_NpSpCU5bwhmICtaR4=d7A1(J8>o4^Lp4 zCtdM+-lRTT4Pz`CY)^k%2i^%#Smpv%B-lY=8{$9QlD&hgg^~TwSglt@%X*p-!}Hw+ zpQ_h*dY-gdDXFOzCM+zW#Yxjura7C} zvY78-N^z44EE2s(MDmbh*DPz-N9(uNQSz3>)I}3vWY?V^?pZw_s4ETxs?zWbUV#Oa zptpXiS4UIW;UYKl%%&;O&7ToffLHuP=4gaP_n|=?4(;|aLTu>eAxa|C6CR5A=t}#k z9UaYwn;>{AWV%7C5VuPh6Zk?36fqrR@qK;?ul(M6wAkyE(p;~YnhOY;Y1%Z;hQTF8>-D=()V_3( z^E*J|)TIQhXsqmfWWUCLD6<~U&bOI+X^KQUe(ThA-=I2{|7o12h{QzjOQVMuRdl|c zg0)26B~#p`R;+I0aynx$(f1qzC!<~R3QdDrBzK5CjN_fEBC6=?@dG5ERmHqc-hI*9 zlHRV18bxt7jXF4OsyV=Ai9^0@FgoCnycA`JJkI>3q6*@1R_NRT?qXR1Y%p<(e5XCB=m*dKJ9edT?9Y?KAFnooN7gt zx$CMo;mLwozAVdg*RzfehvkjrtMWVgw(!<>#yFI2G>8T_1)Ps3| zD0_UhH=FkPmJ`0MsUy{W8aBLDq*mR1^87=cwh27nd8JoGW>bgD*3x@yLlmi`I-kOc z^R7#xujv-B_|lxWU?&sb8 z6BGgyJf%?{wY8c|U9)nf*@6Z%7_bE$Fn$OLgoPd4yRyGx>gRR863b8nFi##v>W!uF=n|8kpeP!by(EFI8VWXW%!O7Oy*>4glW8bDh`gB{Kjm87injWoT_6XJ>6=&tPC}XY_M(0u569uP+5!x(J|_ zWDg@&;1T4L(4c3U*9fxUvhNW80>4d;3ntK6t4dr5SiaoBX4$Zmjj6rOLm@lnD$dI{ zz`5XGog)t^k%I-9rP>o%0(V}o9dJci6-6zxCo13@r90W$+CG#*UiRi#9|y4yw#t`+ zN-~*#Vva+avEBk6xHP~OkI5DpJ9Cp;Tvf0iuHrMz-u@Zw^|u-MFJMAEXp^vmiy<>y zidTR!#u-8Zj_^3jEA`zgdsuqJ+F z3ibqn^6pF0YR)|_Z1S&)B3`7;IeY;69=KoIr*d8Eg6SuOt@WdXlM z3ObDZ!b%y$ukxmr#_f9*K)mG>`DC@+J2c@wvCH#ZOJ>KpbSIJF%B}A8Ri_|-zH=OjGw>pZLmgB*ddqVoU zu2NAB?DKt=gCD{%Yt?wsEh2F*%T=4x%m`W;PFI~7JCZ~3_E&E@Y|VR&T7IOE!2dsRJ`lM!swVrm{=p5d|r0wnBuR9|;XilV%dL7mT3@BDZ%WymSS{Sgfrf#fK}K$9Wxns%~FViN+(|B z`aO2PHF={fCcg2&_oq(YyW=1RGdu!>t~(}~@DH=e2vJwgnse1znJ&mp2hxYQ=&WmB z-Fe|3rd{!#g4>%j64=N(!o;Nqqd#W-4)efX(1D!I-w%%d@#y}z{>>PvoaDa({Oe5H zABI1#1t5L=X~ONf;qy74-=@PL**%;3d2al#a|*vr0RS<$U&jAtYT-G~^CtCgq&S5C z{}F#_SwBa4-oN{e(he%9{Y~ThS2yoD%JVM3pHM)Rp}#@-)f0G*^1R^s8ztq{-=h2~ zzdlEKUY7ce(uDmB<$00nIl%J-;x~XC;V*#S>BMu>=TXsb(|VGBm_Cn@o+JEgknr0c u0N5o30R9p(JU9Q>4gSyO78HLn|L1luCkYAS>(4X^5ugB~JR9}TcmD@sUt+@m literal 0 HcmV?d00001 diff --git a/2407webinar/netflow.ipynb b/2407webinar/netflow.ipynb new file mode 100644 index 0000000..dd1301f --- /dev/null +++ b/2407webinar/netflow.ipynb @@ -0,0 +1,384 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Multicommodity Flow Example Using `gurobipy-pandas`\n", + "\n", + "#### Author: Irv Lustig, Optimization Principal, Princeton Consultants\n", + "\n", + "Solve a multi-commodity flow problem. There are multiple products, which can be \n", + "produced in multiple locations, and have to be shipped over a network to other locations.\n", + "Each location may have supply and/or demand for any product. The network may have\n", + "transhipment locations where freight is interchanged. For each arc in the network, there is \n", + "a limited capacity of the total products that can be carried. Each arc also has a product-specific\n", + "cost for shipping one unit of the product on that arc.\n", + "\n", + "This example is based on `netflow.py` that is supplied by Gurobi.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Import necessary libraries\n", + "\n", + "- `IPython.display` is used to improve the display of `pandas` `Series` by converting them to `DataFrame` for output\n", + "- `PyQt5.QtWidgets` allows prompting for a data file\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [], + "source": [ + "from IPython.display import display\n", + "import pandas as pd\n", + "import gurobipy as grb\n", + "import gurobipy_pandas as gppd" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%gui qt\n", + "\n", + "from PyQt5.QtWidgets import QFileDialog\n", + "\n", + "def gui_fname(dir=None):\n", + " \"\"\"Select a file via a dialog and return the file name.\"\"\"\n", + " if dir is None: dir ='./'\n", + " fname = QFileDialog.getOpenFileName(None, \"Select data file...\", \n", + " dir, filter=\"All files (*);; SM Files (*.sm)\")\n", + " return fname[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Get the file from a prompt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "filename = gui_fname()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Read Data using `pandas`\n", + "\n", + "Read in the data from an Excel file. Converts the data into a dictionary of `pandas` `Series`, with the assumption that the last column is the data column." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [], + "source": [ + "raw_data = pd.read_excel(filename, sheet_name=None)\n", + "data = {\n", + " k: df.set_index(df.columns[:-1].to_list())[df.columns[-1]]\n", + " for k, df in raw_data.items()\n", + "}\n", + "for k, v in data.items():\n", + " print(k)\n", + " display(v.to_frame())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Data Model\n", + "\n", + "## Sets\n", + "\n", + "| Notation | Meaning | Table Locations |\n", + "| ---- | --------------------------- | ----------- | \n", + "| $\\mathcal N$ | Set of network nodes | `cost`: Columns `From`, `To`
`capacity`: Columns `From`, `To`
`supply`: Column `Node`
`demand`: Column `Node` |\n", + "| $\\mathcal P$ | Set of products (commodities) | `cost`: Column `Product`
`supply`: Column `Product`
`demand`: Column `Product` |\n", + "| $\\mathcal A$ | Set of arcs $(n_f,n_t)$, $n_f,n_t\\in\\mathcal A | `cost`: Columns `From`, `To` |\n", + "| $\\mathcal P_a$ | Set of products $p\\in\\mathcal P$ that can be carried on arc $a\\in\\mathcal A$ | `cost`: Columns `Product`, `From`, `To` |\n", + "| $\\mathcal A_p$ | Set of arcs $a\\in\\mathcal A$ that can carry product $p\\in\\mathcal P$ | `cost`: Columns `Product`, `From`, `To` |\n", + "\n", + "\n", + "\n", + "## Numerical Input Values\n", + "\n", + "The input data is converted to pandas `Series`, so the name of each `Series` is also the name of the value.\n", + "\n", + "| Notation | Meaning | Table Name/Value Column | Index Columns \n", + "| ---- | --------------------------- | ------ | ---------- |\n", + "| $\\kappa_a$ | Capacity of arc $a\\in\\mathcal A$ | `capacity` | `From`, `To` |\n", + "| $\\pi_{ap}$ | Cost of carrying product $p$ on arc $a\\in\\mathcal A$, $p\\in\\mathcal P_a$, | `cost` | `Product`, `From`, `To` |\n", + "| $\\sigma_{pn}$ | Supply of product $p\\in\\mathcal P$ at node $n\\in\\mathcal N$. Defaults to 0 | `supply` | `Node` |\n", + "| $\\delta_{pn}$ | Demand of product $p\\in\\mathcal P$ at node $n\\in\\mathcal N$. Defaults to 0 | `demand` | `Node` |" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Compute Sets\n", + "\n", + "- The set $\\mathcal P$ of products can appear in any of the tables `supply`, `demand` and `cost` .\n", + "- The set $\\mathcal N$ of nodes can appear in any of the tables `capacity`, `supply`, `demand` and `cost`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "commodities = set(\n", + " pd.concat(\n", + " [\n", + " data[dfname].index.to_frame()[\"Product\"]\n", + " for dfname in [\"supply\", \"demand\", \"cost\"]\n", + " ]\n", + " ).unique()\n", + ")\n", + "commodities" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "nodes = set(\n", + " pd.concat(\n", + " [\n", + " data[dfname].index.to_frame()[fromto].rename(\"Node\")\n", + " for dfname in [\"capacity\", \"cost\"]\n", + " for fromto in [\"From\", \"To\"]\n", + " ]\n", + " + [data[dfname].index.to_frame()[\"Node\"] for dfname in [\"supply\", \"demand\"]]\n", + " ).unique()\n", + ")\n", + "\n", + "nodes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Compute the Net Flow for each node\n", + "\n", + "The net flow $\\mu_{pn}$ for each product $p\\in\\mathcal P$ and node $n\\in\\mathcal N$ is the sum of the supply less the demand. For transshipment nodes, this value is 0. This is called `inflow` in the code." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "inflow = pd.concat(\n", + " [\n", + " data[\"supply\"].rename(\"net\"),\n", + " data[\"demand\"].rename(\"net\") * -1,\n", + " pd.Series(\n", + " 0,\n", + " index=pd.MultiIndex.from_product(\n", + " [commodities, nodes], names=[\"Product\", \"Node\"]\n", + " ),\n", + " name=\"net\",\n", + " ),\n", + " ]\n", + ").groupby([\"Product\", \"Node\"]).sum()\n", + "inflow" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create the Gurobi Model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "m = grb.Model(\"netflow\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model\n", + "\n", + "### Decision Variables\n", + "\n", + "The model will have one set of decision variables:\n", + "- $X_{pa}$ for $p\\in\\mathcal P$, $a\\in\\mathcal A_p$ represents the amount shipped of product $p$ on arc $a$. We will call this variable `flow` in the code.\n", + "\n", + "The cost of shipment is $\\pi_{ap}$. \n", + "\n", + "This defines the objective function:\n", + "$$\n", + "\\text{minimize}\\quad\\sum_{a\\in\\mathcal A}\\sum_{p\\in\\mathcal P_a} \\pi_{ap}X_{pa}\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "flow = gppd.add_vars(m, data[\"cost\"], obj=data[\"cost\"], name=\"flow\")\n", + "m.update()\n", + "flow" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Constraints\n", + "\n", + "#### Flow on each arc is capacitated\n", + "\n", + "$$\n", + "\\sum_{p\\in\\mathcal P_a} X_{pa} \\le \\kappa_a\\qquad\\forall a\\in\\mathcal A\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "capct = pd.concat(\n", + " [flow.groupby([\"From\", \"To\"]).agg(grb.quicksum), data[\"capacity\"]], axis=1\n", + ").gppd.add_constrs(m, \"flow <= capacity\", name=\"cap\")\n", + "m.update()\n", + "capct" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Conservation of Flow\n", + "\n", + "For each node and each product, the flow out of the node, less the flow into the node is equal to the net flow.\n", + "\n", + "$$\n", + "\\sum_{(n, n_t)\\in A_p} X_{p(n,n_t)} - \\sum_{(n_f, n)} X_{p(n_f,n)} = \\mu_{pn}\\qquad\\forall p\\in\\mathcal P, n\\in\\mathcal N\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "flowct = pd.concat(\n", + " [\n", + " flow.rename_axis(index={\"From\": \"Node\"})\n", + " .groupby([\"Product\", \"Node\"])\n", + " .agg(grb.quicksum)\n", + " .rename(\"flowout\"),\n", + " flow.rename_axis(index={\"To\": \"Node\"})\n", + " .groupby([\"Product\", \"Node\"])\n", + " .agg(grb.quicksum)\n", + " .rename(\"flowin\"),\n", + " inflow,\n", + " ],\n", + " axis=1,\n", + ").fillna(0).gppd.add_constrs(m, \"flowout - flowin == net\", name=\"node\")\n", + "m.update()\n", + "flowct" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Optimize!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "m.optimize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Get the Solution\n", + "\n", + "Only print out arcs with flow, using pandas" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "soln = flow.gppd.X\n", + "soln.to_frame().query(\"flow > 0\").sort_index()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "gurobi1100py311", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 5231a3b75d717a00346d7c350f4340c0d07433f1 Mon Sep 17 00:00:00 2001 From: Simon Bowly Date: Mon, 22 Jul 2024 01:01:40 +1000 Subject: [PATCH 2/7] Convert notebook to myst format --- .../source/examples/data}/mcfdata.v1.xlsx | Bin .../source/examples/data}/mcfdata.v2.xlsx | Bin docs/source/examples/netflow.md | 233 ++++++++++++++++++ 3 files changed, 233 insertions(+) rename {2407webinar => docs/source/examples/data}/mcfdata.v1.xlsx (100%) rename {2407webinar => docs/source/examples/data}/mcfdata.v2.xlsx (100%) create mode 100644 docs/source/examples/netflow.md diff --git a/2407webinar/mcfdata.v1.xlsx b/docs/source/examples/data/mcfdata.v1.xlsx similarity index 100% rename from 2407webinar/mcfdata.v1.xlsx rename to docs/source/examples/data/mcfdata.v1.xlsx diff --git a/2407webinar/mcfdata.v2.xlsx b/docs/source/examples/data/mcfdata.v2.xlsx similarity index 100% rename from 2407webinar/mcfdata.v2.xlsx rename to docs/source/examples/data/mcfdata.v2.xlsx diff --git a/docs/source/examples/netflow.md b/docs/source/examples/netflow.md new file mode 100644 index 0000000..5ddffc6 --- /dev/null +++ b/docs/source/examples/netflow.md @@ -0,0 +1,233 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: 0.13 + jupytext_version: 1.16.3 +kernelspec: + display_name: gppd-etl + language: python + name: gppd-etl +--- + +# Multicommodity Flow Example Using `gurobipy-pandas` + +#### Author: Irv Lustig, Optimization Principal, Princeton Consultants + +Solve a multi-commodity flow problem. There are multiple products, which can be +produced in multiple locations, and have to be shipped over a network to other locations. +Each location may have supply and/or demand for any product. The network may have +transhipment locations where freight is interchanged. For each arc in the network, there is +a limited capacity of the total products that can be carried. Each arc also has a product-specific +cost for shipping one unit of the product on that arc. + +This example is based on `netflow.py` that is supplied by Gurobi. + ++++ + +#### Import necessary libraries + +- `IPython.display` is used to improve the display of `pandas` `Series` by converting them to `DataFrame` for output +- `openpxyl` is required to read `xlsx` data files + +```{code-cell} ipython3 +--- +slideshow: + slide_type: slide +--- +from IPython.display import display +import pandas as pd +import gurobipy as gp +import gurobipy_pandas as gppd +``` + +#### Get the file from a prompt + +```{code-cell} ipython3 +filename = "data/mcfdata.v1.xlsx" +``` + +#### Read Data using `pandas` + +Read in the data from an Excel file. Converts the data into a dictionary of `pandas` `Series`, with the assumption that the last column is the data column. + +```{code-cell} ipython3 +--- +slideshow: + slide_type: slide +--- +raw_data = pd.read_excel(filename, sheet_name=None) +data = { + k: df.set_index(df.columns[:-1].to_list())[df.columns[-1]] + for k, df in raw_data.items() +} +for k, v in data.items(): + print(k) + display(v.to_frame()) +``` + +# Data Model + +## Sets + +| Notation | Meaning | Table Locations | +| ---- | --------------------------- | ----------- | +| $\mathcal N$ | Set of network nodes | `cost`: Columns `From`, `To`
`capacity`: Columns `From`, `To`
`supply`: Column `Node`
`demand`: Column `Node` | +| $\mathcal P$ | Set of products (commodities) | `cost`: Column `Product`
`supply`: Column `Product`
`demand`: Column `Product` | +| $\mathcal A$ | Set of arcs $(n_f,n_t)$, $n_f,n_t\in\mathcal A | `cost`: Columns `From`, `To` | +| $\mathcal P_a$ | Set of products $p\in\mathcal P$ that can be carried on arc $a\in\mathcal A$ | `cost`: Columns `Product`, `From`, `To` | +| $\mathcal A_p$ | Set of arcs $a\in\mathcal A$ that can carry product $p\in\mathcal P$ | `cost`: Columns `Product`, `From`, `To` | + + + +## Numerical Input Values + +The input data is converted to pandas `Series`, so the name of each `Series` is also the name of the value. + +| Notation | Meaning | Table Name/Value Column | Index Columns +| ---- | --------------------------- | ------ | ---------- | +| $\kappa_a$ | Capacity of arc $a\in\mathcal A$ | `capacity` | `From`, `To` | +| $\pi_{ap}$ | Cost of carrying product $p$ on arc $a\in\mathcal A$, $p\in\mathcal P_a$, | `cost` | `Product`, `From`, `To` | +| $\sigma_{pn}$ | Supply of product $p\in\mathcal P$ at node $n\in\mathcal N$. Defaults to 0 | `supply` | `Node` | +| $\delta_{pn}$ | Demand of product $p\in\mathcal P$ at node $n\in\mathcal N$. Defaults to 0 | `demand` | `Node` | + ++++ + +## Compute Sets + +- The set $\mathcal P$ of products can appear in any of the tables `supply`, `demand` and `cost` . +- The set $\mathcal N$ of nodes can appear in any of the tables `capacity`, `supply`, `demand` and `cost` + +```{code-cell} ipython3 +commodities = set( + pd.concat( + [ + data[dfname].index.to_frame()["Product"] + for dfname in ["supply", "demand", "cost"] + ] + ).unique() +) +commodities +``` + +```{code-cell} ipython3 +nodes = set( + pd.concat( + [ + data[dfname].index.to_frame()[fromto].rename("Node") + for dfname in ["capacity", "cost"] + for fromto in ["From", "To"] + ] + + [data[dfname].index.to_frame()["Node"] for dfname in ["supply", "demand"]] + ).unique() +) + +nodes +``` + +### Compute the Net Flow for each node + +The net flow $\mu_{pn}$ for each product $p\in\mathcal P$ and node $n\in\mathcal N$ is the sum of the supply less the demand. For transshipment nodes, this value is 0. This is called `inflow` in the code. + +```{code-cell} ipython3 +inflow = pd.concat( + [ + data["supply"].rename("net"), + data["demand"].rename("net") * -1, + pd.Series( + 0, + index=pd.MultiIndex.from_product( + [commodities, nodes], names=["Product", "Node"] + ), + name="net", + ), + ] +).groupby(["Product", "Node"]).sum() +inflow +``` + +## Create the Gurobi Model + +```{code-cell} ipython3 +m = gp.Model("netflow") +``` + +## Model + +### Decision Variables + +The model will have one set of decision variables: +- $X_{pa}$ for $p\in\mathcal P$, $a\in\mathcal A_p$ represents the amount shipped of product $p$ on arc $a$. We will call this variable `flow` in the code. + +The cost of shipment is $\pi_{ap}$. + +This defines the objective function: +$$ +\text{minimize}\quad\sum_{a\in\mathcal A}\sum_{p\in\mathcal P_a} \pi_{ap}X_{pa} +$$ + +```{code-cell} ipython3 +flow = gppd.add_vars(m, data["cost"], obj=data["cost"], name="flow") +m.update() +flow +``` + +### Constraints + +#### Flow on each arc is capacitated + +$$ +\sum_{p\in\mathcal P_a} X_{pa} \le \kappa_a\qquad\forall a\in\mathcal A +$$ + +```{code-cell} ipython3 +capct = pd.concat( + [flow.groupby(["From", "To"]).agg(gp.quicksum), data["capacity"]], axis=1 +).gppd.add_constrs(m, "flow <= capacity", name="cap") +m.update() +capct +``` + +#### Conservation of Flow + +For each node and each product, the flow out of the node, less the flow into the node is equal to the net flow. + +$$ +\sum_{(n, n_t)\in A_p} X_{p(n,n_t)} - \sum_{(n_f, n)} X_{p(n_f,n)} = \mu_{pn}\qquad\forall p\in\mathcal P, n\in\mathcal N +$$ + +```{code-cell} ipython3 +flowct = pd.concat( + [ + flow.rename_axis(index={"From": "Node"}) + .groupby(["Product", "Node"]) + .agg(gp.quicksum) + .rename("flowout"), + flow.rename_axis(index={"To": "Node"}) + .groupby(["Product", "Node"]) + .agg(gp.quicksum) + .rename("flowin"), + inflow, + ], + axis=1, +).fillna(0).gppd.add_constrs(m, "flowout - flowin == net", name="node") +m.update() +flowct +``` + +# Optimize! + +```{code-cell} ipython3 +m.optimize() +``` + +# Get the Solution + +Only print out arcs with flow, using pandas + +```{code-cell} ipython3 +soln = flow.gppd.X +soln.to_frame().query("flow > 0").sort_index() +``` From b8ced1f0cfb1c249da6273b018c3dd8cc246838f Mon Sep 17 00:00:00 2001 From: Simon Bowly Date: Mon, 22 Jul 2024 01:01:58 +1000 Subject: [PATCH 3/7] Add openpyxl to examples dependencies --- docs/requirements-examples.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/requirements-examples.txt b/docs/requirements-examples.txt index 6da8591..f36c851 100644 --- a/docs/requirements-examples.txt +++ b/docs/requirements-examples.txt @@ -5,3 +5,4 @@ matplotlib>=3.5 nbconvert>=6.5 scikit-learn>=1.0 gurobipy>=10.0 +openpyxl>=3.1 From 755bd3609bc8d93926e934b4e637de20330a828b Mon Sep 17 00:00:00 2001 From: Simon Bowly Date: Mon, 22 Jul 2024 01:02:21 +1000 Subject: [PATCH 4/7] Add netflow example to index --- docs/source/examples.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/examples.rst b/docs/source/examples.rst index e8d5528..3daeac7 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -11,3 +11,4 @@ All example notebooks and input data files are available :ghsrc:`on Github Date: Mon, 22 Jul 2024 01:04:00 +1000 Subject: [PATCH 5/7] Fix heading levels and formatting --- docs/source/examples/netflow.md | 42 +++++++++++++++------------------ 1 file changed, 19 insertions(+), 23 deletions(-) diff --git a/docs/source/examples/netflow.md b/docs/source/examples/netflow.md index 5ddffc6..e1dd474 100644 --- a/docs/source/examples/netflow.md +++ b/docs/source/examples/netflow.md @@ -5,11 +5,7 @@ jupytext: extension: .md format_name: myst format_version: 0.13 - jupytext_version: 1.16.3 -kernelspec: - display_name: gppd-etl - language: python - name: gppd-etl + jupytext_version: 1.14.1 --- # Multicommodity Flow Example Using `gurobipy-pandas` @@ -32,7 +28,7 @@ This example is based on `netflow.py` that is supplied by Gurobi. - `IPython.display` is used to improve the display of `pandas` `Series` by converting them to `DataFrame` for output - `openpxyl` is required to read `xlsx` data files -```{code-cell} ipython3 +```{code-cell} --- slideshow: slide_type: slide @@ -45,7 +41,7 @@ import gurobipy_pandas as gppd #### Get the file from a prompt -```{code-cell} ipython3 +```{code-cell} filename = "data/mcfdata.v1.xlsx" ``` @@ -53,7 +49,7 @@ filename = "data/mcfdata.v1.xlsx" Read in the data from an Excel file. Converts the data into a dictionary of `pandas` `Series`, with the assumption that the last column is the data column. -```{code-cell} ipython3 +```{code-cell} --- slideshow: slide_type: slide @@ -68,9 +64,9 @@ for k, v in data.items(): display(v.to_frame()) ``` -# Data Model +## Data Model -## Sets +### Sets | Notation | Meaning | Table Locations | | ---- | --------------------------- | ----------- | @@ -82,7 +78,7 @@ for k, v in data.items(): -## Numerical Input Values +### Numerical Input Values The input data is converted to pandas `Series`, so the name of each `Series` is also the name of the value. @@ -95,12 +91,12 @@ The input data is converted to pandas `Series`, so the name of each `Series` is +++ -## Compute Sets +### Compute Sets - The set $\mathcal P$ of products can appear in any of the tables `supply`, `demand` and `cost` . - The set $\mathcal N$ of nodes can appear in any of the tables `capacity`, `supply`, `demand` and `cost` -```{code-cell} ipython3 +```{code-cell} commodities = set( pd.concat( [ @@ -112,7 +108,7 @@ commodities = set( commodities ``` -```{code-cell} ipython3 +```{code-cell} nodes = set( pd.concat( [ @@ -131,7 +127,7 @@ nodes The net flow $\mu_{pn}$ for each product $p\in\mathcal P$ and node $n\in\mathcal N$ is the sum of the supply less the demand. For transshipment nodes, this value is 0. This is called `inflow` in the code. -```{code-cell} ipython3 +```{code-cell} inflow = pd.concat( [ data["supply"].rename("net"), @@ -150,7 +146,7 @@ inflow ## Create the Gurobi Model -```{code-cell} ipython3 +```{code-cell} m = gp.Model("netflow") ``` @@ -168,7 +164,7 @@ $$ \text{minimize}\quad\sum_{a\in\mathcal A}\sum_{p\in\mathcal P_a} \pi_{ap}X_{pa} $$ -```{code-cell} ipython3 +```{code-cell} flow = gppd.add_vars(m, data["cost"], obj=data["cost"], name="flow") m.update() flow @@ -182,7 +178,7 @@ $$ \sum_{p\in\mathcal P_a} X_{pa} \le \kappa_a\qquad\forall a\in\mathcal A $$ -```{code-cell} ipython3 +```{code-cell} capct = pd.concat( [flow.groupby(["From", "To"]).agg(gp.quicksum), data["capacity"]], axis=1 ).gppd.add_constrs(m, "flow <= capacity", name="cap") @@ -198,7 +194,7 @@ $$ \sum_{(n, n_t)\in A_p} X_{p(n,n_t)} - \sum_{(n_f, n)} X_{p(n_f,n)} = \mu_{pn}\qquad\forall p\in\mathcal P, n\in\mathcal N $$ -```{code-cell} ipython3 +```{code-cell} flowct = pd.concat( [ flow.rename_axis(index={"From": "Node"}) @@ -217,17 +213,17 @@ m.update() flowct ``` -# Optimize! +## Optimize! -```{code-cell} ipython3 +```{code-cell} m.optimize() ``` -# Get the Solution +## Get the Solution Only print out arcs with flow, using pandas -```{code-cell} ipython3 +```{code-cell} soln = flow.gppd.X soln.to_frame().query("flow > 0").sort_index() ``` From dbb29fb927f55abc49ae0bc8a1393ad3f7429278 Mon Sep 17 00:00:00 2001 From: Simon Bowly Date: Mon, 22 Jul 2024 01:13:52 +1000 Subject: [PATCH 6/7] Fix formatting bug, update title --- docs/source/examples/netflow.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/examples/netflow.md b/docs/source/examples/netflow.md index e1dd474..c238f86 100644 --- a/docs/source/examples/netflow.md +++ b/docs/source/examples/netflow.md @@ -8,9 +8,9 @@ jupytext: jupytext_version: 1.14.1 --- -# Multicommodity Flow Example Using `gurobipy-pandas` +# Multicommodity Flow -#### Author: Irv Lustig, Optimization Principal, Princeton Consultants +**Author**: Irv Lustig, Optimization Principal, Princeton Consultants Solve a multi-commodity flow problem. There are multiple products, which can be produced in multiple locations, and have to be shipped over a network to other locations. @@ -72,7 +72,7 @@ for k, v in data.items(): | ---- | --------------------------- | ----------- | | $\mathcal N$ | Set of network nodes | `cost`: Columns `From`, `To`
`capacity`: Columns `From`, `To`
`supply`: Column `Node`
`demand`: Column `Node` | | $\mathcal P$ | Set of products (commodities) | `cost`: Column `Product`
`supply`: Column `Product`
`demand`: Column `Product` | -| $\mathcal A$ | Set of arcs $(n_f,n_t)$, $n_f,n_t\in\mathcal A | `cost`: Columns `From`, `To` | +|$\mathcal A$ | Set of arcs $(n_f,n_t)$, $n_f,n_t\in\mathcal A$ | `cost`: Columns `From`, `To` | | $\mathcal P_a$ | Set of products $p\in\mathcal P$ that can be carried on arc $a\in\mathcal A$ | `cost`: Columns `Product`, `From`, `To` | | $\mathcal A_p$ | Set of arcs $a\in\mathcal A$ that can carry product $p\in\mathcal P$ | `cost`: Columns `Product`, `From`, `To` | From 0d7a346702a58400ac644d5a33b142036312a5c8 Mon Sep 17 00:00:00 2001 From: Simon Bowly Date: Mon, 22 Jul 2024 01:18:33 +1000 Subject: [PATCH 7/7] Add missed dependency --- docs/requirements.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/requirements.txt b/docs/requirements.txt index 7eb12be..6c09d96 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -10,6 +10,7 @@ myst-parser==0.19.2 nbconvert==7.2.5 nbsphinx==0.8.9 numpydoc==1.6.0 +openpyxl>=3.1 scikit-learn==1.5.0 sphinx-copybutton==0.5.2 sphinx-tabs==3.4.5