-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathatom.xml
522 lines (312 loc) · 277 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>逸思杂陈</title>
<icon>https://www.gravatar.com/avatar/071e91e4d2b41eefac3330385d124f95</icon>
<subtitle>人类一思考，上帝就发笑。</subtitle>
<link href="/atom.xml" rel="self"/>
<link href="http://ponder.work/"/>
<updated>2024-07-12T01:27:28.094Z</updated>
<id>http://ponder.work/</id>
<author>
<name>Jay.Run</name>
<email>[email protected]</email>
</author>
<generator uri="https://hexo.io/">Hexo</generator>
<entry>
<title>跨域的那些事</title>
<link href="http://ponder.work/2023/08/15/cross-domain-things/"/>
<id>http://ponder.work/2023/08/15/cross-domain-things/</id>
<published>2023-08-15T07:00:00.000Z</published>
<updated>2024-07-12T01:27:28.094Z</updated>
<content type="html"><![CDATA[<ul><li>什么是跨域？<ul><li>就是当前域访问了非本域的资源。对于http来说，url代表资源，也就是访问了非本域的url。<a id="more"></a></li></ul></li><li>域的定义是啥？<ul><li>这里的域，也就是同源策略（Same-origin policy）中的源。</li><li>如果两个 URL 的协议、端口和主机都相同的话，则这两个 URL 是同源的。</li><li>当前域：当前网页的域；目标域：访问的资源所在的域</li></ul></li><li><p>为什么要限制跨域请求？信息泄露</p><ul><li>js直接读取浏览器的目标域的sessionid、cookie，伪造请求。<ul><li>比如，当前站点是恶意站点，它用js请求银行站点，盗取信息。</li></ul></li><li><p>CSRF，如img标签的src会被访问，利用受害者已认证的身份，在不知情的情况下向受害者认证的站点发起恶意请求。</p><ul><li>通过在src里构造url，恶意请求目标域<figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><img <span class="attribute">src</span>=<span class="string">"https://bank.com/transfer?amount=1000&to=attackerAccount"</span> </span><br><span class="line"> <span class="attribute">style</span>=<span class="string">"display:none;"</span>></span><br></pre></td></tr></table></figure></li></ul></li><li><p>XSS跨站脚本(Cross-site scripting)注入，导致用户在信息泄露</p><ul><li>一般是接受了用户构造的输入，输入里包含恶意脚本内容，内容未被转义，又输出在页面上，从而被执行</li><li>其他用户查看了该页面，注入的脚本被执行</li></ul></li></ul></li><li>怎么限制跨域？浏览器的同源策略<ul><li>禁止的<ul><li>DOM 同源策略： 不同源的dom之间不能相互操作，多个iframe的情况</li><li>XMLHttpRequest 同源策略： 禁止请求不同源url</li><li>Cookie、LocalStorage、IndexedDB 等存储性内容同源策略</li></ul></li><li>允许的<ul><li>页面中的链接，重定向以及表单提交</li><li><code><script>、<img>、<link></code>这些包含 src 属性的标签可以加载跨域资源。（只能GET）</li></ul></li></ul></li><li>限制跨域导致哪些不便？<ul><li>前后端分离开发时，localhost不能正常访问后端资源</li><li>一些公共的api不能被访问</li><li>https的页面的http的静态资源，不能加载</li></ul></li><li><p>如何绕过同源策略？</p><ul><li>浏览器启动参数（在用户端操作）</li><li>反向代理（在当前域操作）</li><li>JSONP（在目标域操作）<ul><li>利用<code><script></code>允许跨域的特点，设置标签的src为目标域，动态生成需要的javascript内容</li></ul></li><li>跨源资源共享（CORS）（目标域操作）<ul><li>设置相应的reponse header<figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Access-Control-Allow-<span class="string">Origin:</span> <span class="string">https:</span><span class="comment">//foo.example // 所允许的来源域</span></span><br><span class="line">Access-Control-Allow-<span class="string">Methods:</span> POST, GET, OPTIONS <span class="comment">// 所允许的请求方法</span></span><br><span class="line">Access-Control-Allow-<span class="string">Headers:</span> X-PINGOTHER, Content-Type <span class="comment">// 所允许的请求header</span></span><br><span class="line">Access-Control-Max-<span class="string">Age:</span> <span class="number">86400</span></span><br></pre></td></tr></table></figure></li></ul></li></ul></li><li><p>参考</p><ul><li><a href="https://developer.mozilla.org/zh-CN/docs/Web/Security/Same-origin_policy" target="_blank" rel="noopener">https://developer.mozilla.org/zh-CN/docs/Web/Security/Same-origin_policy</a></li><li><a href="https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS" target="_blank" rel="noopener">https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS</a></li><li><a href="https://developer.mozilla.org/zh-CN/docs/Glossary/CSRF" target="_blank" rel="noopener">https://developer.mozilla.org/zh-CN/docs/Glossary/CSRF</a></li><li><a href="https://juejin.cn/post/6879360544323665928" target="_blank" rel="noopener">https://juejin.cn/post/6879360544323665928</a></li><li><a href="https://juejin.cn/post/6867096987804794888" target="_blank" rel="noopener">https://juejin.cn/post/6867096987804794888</a></li></ul></li></ul>]]></content>
<summary type="html">
<ul>
<li>什么是跨域？<ul>
<li>就是当前域访问了非本域的资源。对于http来说，url代表资源，也就是访问了非本域的url。
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Web" scheme="http://ponder.work/tags/Web/"/>
</entry>
<entry>
<title>HomeBrew 与无 root 权限 Linux 环境包管理</title>
<link href="http://ponder.work/2023/05/28/homebrew-as-non-root-package-manager/"/>
<id>http://ponder.work/2023/05/28/homebrew-as-non-root-package-manager/</id>
<published>2023-05-28T12:00:00.000Z</published>
<updated>2023-05-28T15:48:25.000Z</updated>
<content type="html"><![CDATA[<p>一些公用的 Linux 服务器，处于维护以及安全考虑，一般只会提供普通权限用户给使用者。<br>普通用户的权限满足日常使用是够了，但是难以配置自己的开发环境，安装一些自己需要的包。</p><p>如果都从源码编译安装软件，依赖的维护过于复杂，初始编译工具链的版本可能也不满足需求，如 gcc 版本过低。<br>如果申请 sudo 权限或者请求更新系统或安装 docker，后期责任难以界定，运维和管理员一般也不会同意。</p><p>所以，最优方案还是有需求的用户在个人目录维护自己的工具链和环境。下文方案为围绕 HomeBrew 构建。<br><a id="more"></a></p><h2 id="安装-miniconda-解决前置依赖"><a href="#安装-miniconda-解决前置依赖" class="headerlink" title="安装 miniconda 解决前置依赖"></a>安装 miniconda 解决前置依赖</h2><p>如果你的系统比较新，可以直接尝试<code>安装 HomeBrew</code>。</p><p>基于上面讨论的内容，公用服务器一般存在系统版本低的问题，是 centos7 或者 centos6 也毫不稀奇，而且如 glibc 等库的版本也非常低。</p><p>安装 HomeBrew 有两个强依赖，git 及 curl，而且依赖的版本都比较高，centos7 的版本也不能满足。<br>另外，由于 Brew 不少软件都需要从源码编译，gcc 和良好的网络环境也不可缺少。</p><p>幸好 miniconda 能够解决以上几点问题。miniconda 只是提供 HomeBrew 安装的依赖，后续可以删除。</p><p>配置 conda 源（可选）： 新建 <code>.condarc</code>，包含以下内容</p><figure class="highlight less"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">channels</span>:</span><br><span class="line"> - defaults</span><br><span class="line"><span class="attribute">show_channel_urls</span>: true</span><br><span class="line"><span class="attribute">default_channels</span>:</span><br><span class="line"> - <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main</span></span><br><span class="line"> - <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r</span></span><br><span class="line"> - <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2</span></span><br><span class="line"><span class="attribute">custom_channels</span>:</span><br><span class="line"> <span class="attribute">conda-forge</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br><span class="line"> <span class="attribute">msys2</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br><span class="line"> <span class="attribute">bioconda</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br><span class="line"> <span class="attribute">menpo</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br><span class="line"> <span class="attribute">pytorch</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br><span class="line"> <span class="attribute">pytorch-lts</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br><span class="line"> <span class="attribute">simpleitk</span>: <span class="attribute">https</span>:<span class="comment">//mirrors.tuna.tsinghua.edu.cn/anaconda/cloud</span></span><br></pre></td></tr></table></figure><p>下载及安装 miniconda</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 下载</span></span><br><span class="line"><span class="meta">#</span><span class="bash"> 如果服务器的内置证书已过期， 增加 --no-check-certificate 条件跳过证书验证</span></span><br><span class="line">wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && chmod +x Miniconda3-latest-Linux-x86_64.sh</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 安装</span></span><br><span class="line">./Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3</span><br><span class="line">source ~/miniconda3/etc/profile.d/conda.sh</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 安装所需包</span></span><br><span class="line">conda install -y gcc_linux-64 gxx_linux-64 curl git</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 链接 gcc，等 HomeBrew 安装完成这些链接可以删掉</span></span><br><span class="line">cd ~/miniconda3/bin/</span><br><span class="line">ln -s x86_64-conda_cos6-linux-gnu-gcc gcc</span><br><span class="line">ln -s x86_64-conda_cos6-linux-gnu-cpp c++</span><br><span class="line">ln -s gcc cc</span><br></pre></td></tr></table></figure><h2 id="配置安装环境变量"><a href="#配置安装环境变量" class="headerlink" title="配置安装环境变量"></a>配置安装环境变量</h2><p>这些环境变量也可以配置到 bashrc 等文件，使之永久生效</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 设置 curl 和 git，可选</span></span><br><span class="line">export HOMEBREW_CURL_PATH=~/miniconda3/bin/curl</span><br><span class="line">export HOMEBREW_GIT_PATH=~/miniconda3/bin/git</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 设置安装源为清华源，如果网络畅通可忽略，清华源也可能403</span></span><br><span class="line">export HOMEBREW_INSTALL_FROM_API=1</span><br><span class="line">export HOMEBREW_API_DOMAIN="https://mirrors.tuna.tsinghua.edu.cn/homebrew-bottles/api"</span><br><span class="line">export HOMEBREW_BOTTLE_DOMAIN="https://mirrors.tuna.tsinghua.edu.cn/homebrew-bottles"</span><br><span class="line">export HOMEBREW_BREW_GIT_REMOTE="https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/brew.git"</span><br><span class="line">export HOMEBREW_CORE_GIT_REMOTE="https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/homebrew-core.git"</span><br></pre></td></tr></table></figure><h2 id="安装-HomeBrew"><a href="#安装-HomeBrew" class="headerlink" title="安装 HomeBrew"></a>安装 HomeBrew</h2><p>由于没有 root 权限，HomeBrew 需要手动安装。<br>由于是手动安装，位置与默认安装位置不同，很多预编译的包就不能用了，都得从源码编译，所以网络和机器性能以及耐心很重要。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 下载，也可使用清华源 https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/brew.git</span></span><br><span class="line">git clone https://github.com/Homebrew/brew ~/.homebrew</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 安装</span></span><br><span class="line">eval "$(~/.homebrew/bin/brew shellenv)"</span><br><span class="line">brew update --force --quiet</span><br><span class="line">chmod -R go-w "$(brew --prefix)/share/zsh"</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 自动加载 brew</span></span><br><span class="line">echo 'eval "$(~/.homebrew/bin/brew shellenv)"' >> ~/.bashrc</span><br></pre></td></tr></table></figure><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://docs.brew.sh/Installation" target="_blank" rel="noopener">https://docs.brew.sh/Installation</a></li><li><a href="https://mirrors.tuna.tsinghua.edu.cn/help/anaconda/" target="_blank" rel="noopener">https://mirrors.tuna.tsinghua.edu.cn/help/anaconda/</a></li><li><a href="https://mirrors.tuna.tsinghua.edu.cn/help/homebrew/" target="_blank" rel="noopener">https://mirrors.tuna.tsinghua.edu.cn/help/homebrew/</a></li><li><a href="https://github.com/tuna/issues/issues/1353" target="_blank" rel="noopener">https://github.com/tuna/issues/issues/1353</a></li></ul>]]></content>
<summary type="html">
<p>一些公用的 Linux 服务器，处于维护以及安全考虑，一般只会提供普通权限用户给使用者。<br>普通用户的权限满足日常使用是够了，但是难以配置自己的开发环境，安装一些自己需要的包。</p>
<p>如果都从源码编译安装软件，依赖的维护过于复杂，初始编译工具链的版本可能也不满足需求，如 gcc 版本过低。<br>如果申请 sudo 权限或者请求更新系统或安装 docker，后期责任难以界定，运维和管理员一般也不会同意。</p>
<p>所以，最优方案还是有需求的用户在个人目录维护自己的工具链和环境。下文方案为围绕 HomeBrew 构建。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Linux" scheme="http://ponder.work/tags/Linux/"/>
</entry>
<entry>
<title>给 macOS 词典增加生词本功能</title>
<link href="http://ponder.work/2022/11/12/add-wordlist-for-macOS-dictionary/"/>
<id>http://ponder.work/2022/11/12/add-wordlist-for-macOS-dictionary/</id>
<published>2022-11-12T12:00:00.000Z</published>
<updated>2024-07-09T13:50:24.797Z</updated>
<content type="html"><![CDATA[<p>macOS 系统的自带词典应用非常强大，与其他应用整合很好，快捷取词很方便（command+control+d）。<br>但是美中不足的是缺少生词本功能，查了单词又很容易忘记，对语言学习者来说就有些不便了。</p><p>经过本强迫症的探索，终于找到基于 Karabiner-Elements + Automator + Logseq 的完美生词本方案。<br>最后的效果是，快捷键取词的同时记录单词卡片到Logseq对应的笔记。<br><a id="more"></a></p><h2 id="词典词库扩充"><a href="#词典词库扩充" class="headerlink" title="词典词库扩充"></a>词典词库扩充</h2><p>参考<a href="https://zhuanlan.zhihu.com/p/433646737" target="_blank" rel="noopener">知乎文章</a>安装好《朗道英汉字典5.0》<br>这是为了有个释义简洁的词典，方便后续生成生词本词条</p><p><img src="https://image.ponder.work/mweb/2022-11-02-16673997943438.jpg" alt></p><h2 id="编写workflow"><a href="#编写workflow" class="headerlink" title="编写workflow"></a>编写workflow</h2><p>使用 macOS 自带应用 Automator（自动操作）编写workflow，将当前鼠标所在位置的文本提取并保存制卡。</p><p>首先打开 Automator.app 新建一个 Quick Aciont(快速操作)<br><img src="https://image.ponder.work/mweb/2022-11-02-16673999123628.jpg" alt><br>然后依次拖入“获得词语定义”，“运行Shell脚本”等步骤，并调整如下几个位置的选项。<br><img src="https://image.ponder.work/mweb/2022-11-02-16674004811769.jpg" alt></p><p>修改脚本里的代码为如下内容，生词本路径相应替换，并相应位置新建好生词本文件。<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># -*- coding:utf-8 -*-</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">from</span> __future__ <span class="keyword">import</span> unicode_literals, print_function</span><br><span class="line"><span class="keyword">import</span> sys, os, io, subprocess</span><br><span class="line"></span><br><span class="line">FILE=os.path.expanduser(<span class="string">"~/weiyun_sync/!sync/logseq-note/pages/生词本.md"</span>)</span><br><span class="line">output = []</span><br><span class="line">text = sys.argv[<span class="number">1</span>].decode(<span class="string">'utf8'</span>) <span class="keyword">if</span> sys.version_info.major == <span class="number">2</span> <span class="keyword">else</span> sys.argv[<span class="number">1</span>]</span><br><span class="line"></span><br><span class="line">lines = [i.strip() <span class="keyword">for</span> i <span class="keyword">in</span> text.splitlines() <span class="keyword">if</span> i.strip()]</span><br><span class="line"><span class="keyword">if</span> len(lines) < <span class="number">2</span>:</span><br><span class="line"> exit(<span class="number">0</span>)</span><br><span class="line"></span><br><span class="line">word = lines[<span class="number">0</span>]</span><br><span class="line"><span class="keyword">if</span> lines[<span class="number">1</span>][<span class="number">0</span>] == <span class="string">'*'</span>:</span><br><span class="line"> output.append(<span class="string">'- {}\t{} [[card]]'</span>.format(word, lines[<span class="number">1</span>]))</span><br><span class="line"> lines = lines[<span class="number">2</span>:]</span><br><span class="line"><span class="keyword">else</span>:</span><br><span class="line"> output.append(<span class="string">'- {}\t [[card]]'</span>.format(word))</span><br><span class="line"> lines = lines[<span class="number">1</span>:]</span><br><span class="line">output.append(<span class="string">'\t- {}'</span>.format(lines[<span class="number">0</span>]))</span><br><span class="line"><span class="keyword">for</span> line <span class="keyword">in</span> lines[<span class="number">1</span>:]:</span><br><span class="line"> output.append(<span class="string">'\t '</span> + line)</span><br><span class="line"></span><br><span class="line">old_words = set()</span><br><span class="line"><span class="keyword">with</span> io.open(FILE, <span class="string">'r'</span>, encoding=<span class="string">'utf8'</span>) <span class="keyword">as</span> fp:</span><br><span class="line"> <span class="keyword">for</span> line <span class="keyword">in</span> fp:</span><br><span class="line"> parts = line.split()</span><br><span class="line"> <span class="keyword">if</span> line.startswith(<span class="string">'-'</span>) <span class="keyword">and</span> len(parts) > <span class="number">1</span>:</span><br><span class="line"> old_words.add(parts[<span class="number">1</span>])</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> word <span class="keyword">not</span> <span class="keyword">in</span> old_words:</span><br><span class="line"> <span class="keyword">with</span> io.open(FILE, <span class="string">'a'</span>, encoding=<span class="string">'utf8'</span>) <span class="keyword">as</span> fp:</span><br><span class="line"> fp.write(<span class="string">'\n'</span>)</span><br><span class="line"> fp.write(<span class="string">'\n'</span>.join(output))</span><br><span class="line"> fp.write(<span class="string">'\n'</span>)</span><br><span class="line"> subprocess.check_call([<span class="string">'osascript'</span>, <span class="string">'-e'</span>, <span class="string">u'display notification "添加 {}" with title "生词本"'</span>.format(word)])</span><br><span class="line"><span class="keyword">else</span>:</span><br><span class="line"> subprocess.check_call([<span class="string">'osascript'</span>, <span class="string">'-e'</span>, <span class="string">u'display notification "跳过 {}" with title "生词本"'</span>.format(word)])</span><br></pre></td></tr></table></figure></p><p>选择路径保存好 workflow，然后在 <code>键盘 - 快捷键 - 服务</code> 中能看到新建的workflow。<br>为它设置快捷键 <code>command + shift + alt + 1</code><br><img src="https://image.ponder.work/mweb/2022-11-02-16674009451584.jpg" alt></p><h2 id="Karabiner-Elements"><a href="#Karabiner-Elements" class="headerlink" title="Karabiner-Elements"></a>Karabiner-Elements</h2><p><a href="https://karabiner-elements.pqrs.org/" target="_blank" rel="noopener">Karabiner-Elements</a> 是 macOS 平台的一个重新映射快捷键的软件。<br>这里我们使用它将“查询单词”和“触发workflow”整合在一起，当然它还支持很多用途，这里就不赘述了。</p><p>注意确保Karabiner相关权限，并且设置中下图相关设备是勾选状态</p><p><img src="https://image.ponder.work/mweb/2022-11-04-16675619005392.jpg" alt></p><p>安装好Karabiner-Elements后，打开它的配置文件<br>路径在 <code>/Users/<用户名>/.config/karabiner/karabiner.json</code></p><p>在 <code>profiles -> complex_modifications -> rules</code> 列表中增加一项配置，内容如下。<br>然后保存，Karabiner会自动加载新的配置。</p><p>这里是将鼠标的侧键（靠前的）映射为查单词的快捷键，实现一键查词。<br>也可以根据需要更改按键，通过<a href="https://karabiner-elements.pqrs.org/docs/manual/operation/eventviewer/" target="_blank" rel="noopener">EventViewer</a>可以查看按键代码，配置文件格式可参考<a href="https://karabiner-elements.pqrs.org/docs/json/complex-modifications-manipulator-definition/to/" target="_blank" rel="noopener">官方文档</a></p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> <span class="attr">"description"</span>: <span class="string">"Mouse"</span>,</span><br><span class="line"> <span class="attr">"manipulators"</span>: [</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"from"</span>: {</span><br><span class="line"> <span class="attr">"pointing_button"</span>: <span class="string">"button5"</span></span><br><span class="line"> },</span><br><span class="line"> <span class="attr">"to"</span>: [</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"pointing_button"</span>: <span class="string">"button1"</span></span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"pointing_button"</span>: <span class="string">"button1"</span></span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"key_code"</span>: <span class="string">"d"</span>,</span><br><span class="line"> <span class="attr">"modifiers"</span>: [</span><br><span class="line"> <span class="string">"left_command"</span>,</span><br><span class="line"> <span class="string">"left_control"</span></span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"key_code"</span>: <span class="string">"1"</span>,</span><br><span class="line"> <span class="attr">"modifiers"</span>: [</span><br><span class="line"> <span class="string">"left_option"</span>,</span><br><span class="line"> <span class="string">"left_shift"</span>,</span><br><span class="line"> <span class="string">"left_command"</span></span><br><span class="line"> ]</span><br><span class="line"> }</span><br><span class="line"> ],</span><br><span class="line"> <span class="attr">"type"</span>: <span class="string">"basic"</span></span><br><span class="line"> }</span><br><span class="line"> ]</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><img src="https://image.ponder.work/mweb/2022-11-02-16674017788245.jpg" alt></p><h2 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h2><p>可以看到 Logseq 中卡片生成的效果<br><img src="https://image.ponder.work/mweb/2022-11-02-16674007610777.jpg" alt><br><img src="https://image.ponder.work/mweb/2022-11-02-16674023982597.jpg" alt></p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://karabiner-elements.pqrs.org/docs/" target="_blank" rel="noopener">https://karabiner-elements.pqrs.org/docs/</a></li><li><a href="https://zhuanlan.zhihu.com/p/433646737" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/433646737</a></li><li><a href="https://hectorguo.com/zh/save-words-in-dictionary/" target="_blank" rel="noopener">https://hectorguo.com/zh/save-words-in-dictionary/</a></li><li><a href="https://github.com/jjgod/mac-dictionary-kit" target="_blank" rel="noopener">https://github.com/jjgod/mac-dictionary-kit</a></li><li><a href="https://lightcss.com/mac-dictionary/" target="_blank" rel="noopener">https://lightcss.com/mac-dictionary/</a></li></ul>]]></content>
<summary type="html">
<p>macOS 系统的自带词典应用非常强大，与其他应用整合很好，快捷取词很方便（command+control+d）。<br>但是美中不足的是缺少生词本功能，查了单词又很容易忘记，对语言学习者来说就有些不便了。</p>
<p>经过本强迫症的探索，终于找到基于 Karabiner-Elements + Automator + Logseq 的完美生词本方案。<br>最后的效果是，快捷键取词的同时记录单词卡片到Logseq对应的笔记。<br>
</summary>
<category term="工作生活" scheme="http://ponder.work/categories/%E5%B7%A5%E4%BD%9C%E7%94%9F%E6%B4%BB/"/>
<category term="mac" scheme="http://ponder.work/tags/mac/"/>
</entry>
<entry>
<title>关闭子进程打开的文件描述符</title>
<link href="http://ponder.work/2022/08/30/close-subprocess-opened-fd/"/>
<id>http://ponder.work/2022/08/30/close-subprocess-opened-fd/</id>
<published>2022-08-30T14:48:00.000Z</published>
<updated>2022-11-02T15:28:48.000Z</updated>
<content type="html"><![CDATA[<p>我们在测试代码时，由于需要经常重启服务，经常会发现服务端口被占用。<br>一般kill掉后台进程就ok了，但是如果服务有启动一些常驻的后台程序，可能也会导致端口不能释放。</p><p>在类UNIX系统中，一切被打开的文件、端口被抽象为文件描述符（file descriptor）<br>从python3.4开始，文件描述符默认是non-inheritable，也就是子进程不会共享文件描述符。<br><a id="more"></a></p><h2 id="问题"><a href="#问题" class="headerlink" title="问题"></a>问题</h2><p>一般为了实现多进程、多线程的webserver，服务端口fd必须设置为继承（set_inheritable），这样才能多进程监听一个端口（配合SO_REUSEPORT）<br>典型的是使用flask的测试服务器的场景，这里我们写一段代码模拟。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> socket, os</span><br><span class="line">server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)</span><br><span class="line">server.bind((<span class="string">'127.0.0.1'</span>, <span class="number">22222</span>))</span><br><span class="line">server.set_inheritable(<span class="literal">True</span>)</span><br><span class="line"></span><br><span class="line">os.system(<span class="string">"python -c 'import time;time.sleep(1000)' "</span>)</span><br></pre></td></tr></table></figure><p>我们通过<code>lsof -p {pid}</code>可以看到这两个进程的所有文件描述符<br>server进程, 可以看到服务端口的fd是4<br><figure class="highlight tap"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">COMMAND PID FD TYPE DEVICE SIZE/OFF NODE NAME</span><br><span class="line">ptpython<span class="number"> 6214 </span>cwd DIR 253,0 <span class="number"> 4096 </span><span class="number"> 872946898 </span>/</span><br><span class="line">...</span><br><span class="line">ptpython<span class="number"> 6214 </span> 0u CHR 136,13 0t0 <span class="number"> 16 </span>/dev/pts/13</span><br><span class="line">ptpython<span class="number"> 6214 </span> 1u CHR 136,13 0t0 <span class="number"> 16 </span>/dev/pts/13</span><br><span class="line">ptpython<span class="number"> 6214 </span> 2u CHR 136,13 0t0 <span class="number"> 16 </span>/dev/pts/13</span><br><span class="line">ptpython<span class="number"> 6214 </span> 3r CHR 1,9 0t0 <span class="number"> 2057 </span>/dev/urandom</span><br><span class="line">ptpython<span class="number"> 6214 </span> 4u sock 0,7 0t0 <span class="number"> 58345077 </span>protocol: TCP</span><br><span class="line">ptpython<span class="number"> 6214 </span> 5u a_inode 0,10 <span class="number"> 0 </span> <span class="number"> 8627 </span>[eventpoll]</span><br><span class="line">ptpython<span class="number"> 6214 </span> 6u unix 0x0000000000000000 0t0 <span class="number"> 58368029 </span>socket</span><br><span class="line">ptpython<span class="number"> 6214 </span> 7u unix 0x0000000000000000 0t0 <span class="number"> 58368030 </span>socket</span><br></pre></td></tr></table></figure></p><p>sleep子进程，也拥有fd=4的文件描述符<br><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">COMMAND PID FD <span class="built_in"> TYPE </span>DEVICE SIZE/OFF NODE NAME</span><br><span class="line">python 18022 cwd DIR 253,0 4096 872946898 /</span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line">python 18022 0u CHR 136,13 0t0 16 /dev/pts/13</span><br><span class="line">python 18022 1u CHR 136,13 0t0 16 /dev/pts/13</span><br><span class="line">python 18022 2u CHR 136,13 0t0 16 /dev/pts/13</span><br><span class="line">python 18022 4u sock 0,7 0t0 58345077 protocol: TCP</span><br></pre></td></tr></table></figure></p><p>如果server进程退出时，sleep进程没有退出，fd=4对应的端口就被占用了，服务也就不能正常启动了。</p><h2 id="解决方法"><a href="#解决方法" class="headerlink" title="解决方法"></a>解决方法</h2><h3 id="手动清理"><a href="#手动清理" class="headerlink" title="手动清理"></a>手动清理</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> time</span><br><span class="line"></span><br><span class="line">os.system(<span class="string">f'lsof -p <span class="subst">{os.getpid()}</span>'</span>)</span><br><span class="line">os.closerange(<span class="number">3</span>, <span class="number">100</span>) <span class="comment"># 这里假定打开文件描述符不会超过100</span></span><br><span class="line">time.sleep(<span class="number">5</span>)</span><br><span class="line">os.system(<span class="string">f'lsof -p <span class="subst">{os.getpid()}</span>'</span>)</span><br><span class="line"><span class="comment"># 后面执行需要的业务代码</span></span><br></pre></td></tr></table></figure><h3 id="使用close-fds"><a href="#使用close-fds" class="headerlink" title="使用close_fds"></a>使用close_fds</h3><p>使用subprocess库而不是os来启动子程序， 通过close_fds参数关闭多余的文件描述符<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> subprocess</span><br><span class="line">subprocess.call(<span class="string">"python -c 'import time;time.sleep(1000)'"</span>, shell=<span class="literal">True</span>, close_fds=<span class="literal">True</span>)</span><br></pre></td></tr></table></figure></p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://docs.python.org/3/library/os.html#inheritance-of-file-descriptors" target="_blank" rel="noopener">https://docs.python.org/3/library/os.html#inheritance-of-file-descriptors</a></li><li><a href="https://docs.python.org/3/library/subprocess.html#subprocess.Popen" target="_blank" rel="noopener">https://docs.python.org/3/library/subprocess.html#subprocess.Popen</a></li><li><a href="https://stackoverflow.com/questions/2023608/check-what-files-are-open-in-python#answer-25069136" target="_blank" rel="noopener">https://stackoverflow.com/questions/2023608/check-what-files-are-open-in-python#answer-25069136</a></li></ul>]]></content>
<summary type="html">
<p>我们在测试代码时，由于需要经常重启服务，经常会发现服务端口被占用。<br>一般kill掉后台进程就ok了，但是如果服务有启动一些常驻的后台程序，可能也会导致端口不能释放。</p>
<p>在类UNIX系统中，一切被打开的文件、端口被抽象为文件描述符（file descriptor）<br>从python3.4开始，文件描述符默认是non-inheritable，也就是子进程不会共享文件描述符。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Python" scheme="http://ponder.work/tags/Python/"/>
</entry>
<entry>
<title>容器内进程优雅退出</title>
<link href="http://ponder.work/2022/07/10/gracefully-shutdown-container/"/>
<id>http://ponder.work/2022/07/10/gracefully-shutdown-container/</id>
<published>2022-07-10T13:48:00.000Z</published>
<updated>2022-11-02T15:28:48.000Z</updated>
<content type="html"><![CDATA[<p>在使用 docker 时，常常会碰到进程退出时资源清理的问题，比如保证当前请求处理完成，再退出程序。</p><p>当执行 <code>docker stop xxx</code> 时，docker会向主进程（pid=1）发送 <code>SIGTERM</code> 信号<br>如果在一定时间(默认为10s)内进程没有退出，会进一步发送 <code>SIGKILL</code> 直接杀死程序，该信号既不能被捕捉也不能被忽略。</p><p>一般的web框架或者rpc框架都集成了 <code>SIGTERM</code> 信号处理程序， 一般不用担心优雅退出的问题。<br>但是如果你的容器内有多个程序（称为胖容器，一般不推荐），那么就需要做一些操作保证所有程序优雅退出。</p><a id="more"></a><h2 id="signals"><a href="#signals" class="headerlink" title="signals"></a>signals</h2><p>信号是一种进程间通信机制，它给应用程序提供一种异步的软件中断，使应用程序有机会接受其他程序活终端发送的命令(即信号)。</p><p>应用程序收到信号后，有三种处理方式：忽略，默认，或捕捉。</p><p>常见信号：</p><div class="table-container"><table><thead><tr><th>信号名称</th><th>信号数</th><th>描述</th><th style="text-align:left">默认操作</th></tr></thead><tbody><tr><td>SIGHUP</td><td>1</td><td>当用户退出Linux登录时，前台进程组和后台有对终端输出的进程将会收到SIGHUP信号。对于与终端脱离关系的守护进程，这个信号用于通知它重新读取配置文件。</td><td style="text-align:left">终止进程</td></tr><tr><td>SIGINT</td><td>2</td><td>程序终止(interrupt)信号，在用户键入 Ctrl+C 时发出。</td><td style="text-align:left">终止进程</td></tr><tr><td>SIGQUIT</td><td>3</td><td>和SIGINT类似，但由QUIT字符(通常是Ctrl /)来控制。</td><td style="text-align:left">终止进程并dump core</td></tr><tr><td>SIGFPE</td><td>8</td><td>在发生致命的算术运算错误时发出。不仅包括浮点运算错误，还包括溢出及除数为0等其它所有的算术错误。</td><td style="text-align:left">终止进程并dump core</td></tr><tr><td>SIGKILL</td><td>9</td><td>用来立即结束程序的运行。本信号不能被阻塞，处理和忽略。</td><td style="text-align:left">终止进程</td></tr><tr><td>SIGALRM</td><td>14</td><td>时钟定时信号，计算的是实际的时间或时钟时间。alarm 函数使用该信号。</td><td style="text-align:left">终止进程</td></tr><tr><td>SIGTERM</td><td>15</td><td>通常用来要求程序自己正常退出；kill 命令缺省产生这个信号。</td><td style="text-align:left">终止进程</td></tr></tbody></table></div><h2 id="Dockerfile"><a href="#Dockerfile" class="headerlink" title="Dockerfile"></a>Dockerfile</h2><p>下面以 supervisor 为例，Dockerfile 如下</p><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">FROM</span> centos:centos7</span><br><span class="line"><span class="keyword">ENV</span> PYTHONUNBUFFERED=<span class="number">1</span> TZ=Asia/Shanghai</span><br><span class="line"><span class="keyword">RUN</span><span class="bash"> yum -y install epel-release && \</span></span><br><span class="line"><span class="bash"> yum -y install supervisor && \</span></span><br><span class="line"><span class="bash"> yum -y clean all && rm -rf /var/cache</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">COPY</span><span class="bash"> ./ /root/</span></span><br><span class="line"><span class="keyword">ENTRYPOINT</span><span class="bash"> [ <span class="string">"/usr/bin/supervisord"</span>, <span class="string">"-n"</span>, <span class="string">"-c"</span>, <span class="string">"/etc/supervisord.conf"</span> ]</span></span><br></pre></td></tr></table></figure><h2 id="trap"><a href="#trap" class="headerlink" title="trap"></a>trap</h2><p>正常情况，容器退出时supervisor启动的其他程序并不会收到 <code>SIGTERM</code> 信号，导致子程序直接退出了。</p><p>这里使用 <code>trap</code> 对程序的异常处理进行包装<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">trap</span> <siginal handler> <signal 1> <signal 2> ...</span><br></pre></td></tr></table></figure></p><p>新建一个初始化脚本，<code>init.sh</code><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/sh</span></span><br><span class="line"></span><br><span class="line">/usr/bin/supervisord -n -c /etc/supervisord.conf &</span><br><span class="line"></span><br><span class="line"><span class="built_in">trap</span> <span class="string">"supervisorctl stop all && sleep 3"</span> TERM INT</span><br><span class="line"></span><br><span class="line"><span class="built_in">wait</span></span><br></pre></td></tr></table></figure></p><p>修改 ENTRYPOINT 为如下<br><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">ENTRYPOINT</span><span class="bash"> [<span class="string">"sh"</span>, <span class="string">"/root/init.sh"</span>]</span></span><br></pre></td></tr></table></figure></p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/" target="_blank" rel="noopener">https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/</a></li><li><a href="https://www.cnblogs.com/taobataoma/archive/2007/08/30/875743.html" target="_blank" rel="noopener">https://www.cnblogs.com/taobataoma/archive/2007/08/30/875743.html</a></li><li><a href="https://wangchujiang.com/linux-command/c/trap.html" target="_blank" rel="noopener">https://wangchujiang.com/linux-command/c/trap.html</a></li></ul>]]></content>
<summary type="html">
<p>在使用 docker 时，常常会碰到进程退出时资源清理的问题，比如保证当前请求处理完成，再退出程序。</p>
<p>当执行 <code>docker stop xxx</code> 时，docker会向主进程（pid=1）发送 <code>SIGTERM</code> 信号<br>如果在一定时间(默认为10s)内进程没有退出，会进一步发送 <code>SIGKILL</code> 直接杀死程序，该信号既不能被捕捉也不能被忽略。</p>
<p>一般的web框架或者rpc框架都集成了 <code>SIGTERM</code> 信号处理程序， 一般不用担心优雅退出的问题。<br>但是如果你的容器内有多个程序（称为胖容器，一般不推荐），那么就需要做一些操作保证所有程序优雅退出。</p>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Linux" scheme="http://ponder.work/tags/Linux/"/>
<category term="Docker" scheme="http://ponder.work/tags/Docker/"/>
</entry>
<entry>
<title>Python 循环变量泄露与延迟绑定</title>
<link href="http://ponder.work/2022/03/04/python-loop-variables-leak/"/>
<id>http://ponder.work/2022/03/04/python-loop-variables-leak/</id>
<published>2022-03-04T12:17:00.000Z</published>
<updated>2022-03-05T05:14:52.000Z</updated>
<content type="html"><![CDATA[<p>循环变量泄露与延迟绑定叠加在一起，会产生一些让人迷惑的结果。<br><a id="more"></a></p><h2 id="梦开始的地方"><a href="#梦开始的地方" class="headerlink" title="梦开始的地方"></a>梦开始的地方</h2><p>先看看一开始的问题，可以看到这里lambda函数的返回值一直在变。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">xx = []</span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> [<span class="number">1</span>,<span class="number">2</span>,<span class="number">3</span>]:</span><br><span class="line"> xx.append(<span class="keyword">lambda</span>: i)</span><br><span class="line"></span><br><span class="line">print(<span class="string">'a:'</span>, xx[<span class="number">0</span>]())</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> j <span class="keyword">in</span> xx:</span><br><span class="line"> print(j())</span><br><span class="line"></span><br><span class="line">print(<span class="string">'b:'</span>, xx[<span class="number">0</span>]())</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> xx:</span><br><span class="line"> print(i, i())</span><br><span class="line"></span><br><span class="line">print(<span class="string">'c:'</span>, xx[<span class="number">0</span>], xx[<span class="number">0</span>]())</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> [<span class="number">4</span>, <span class="number">5</span>, <span class="number">6</span>]:</span><br><span class="line"> print(i)</span><br><span class="line"></span><br><span class="line">print(<span class="string">'d:'</span>, xx[<span class="number">0</span>], xx[<span class="number">0</span>]())</span><br></pre></td></tr></table></figure><p>输出如下</p><figure class="highlight dts"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">a:</span> <span class="number">3</span></span><br><span class="line"><span class="number">3</span></span><br><span class="line"><span class="number">3</span></span><br><span class="line"><span class="number">3</span></span><br><span class="line"><span class="symbol">b:</span> <span class="number">3</span></span><br><span class="line"><span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30310</span>> <span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30310</span>></span><br><span class="line"><span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca303a0</span>> <span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca303a0</span>></span><br><span class="line"><span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30430</span>> <span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30430</span>></span><br><span class="line"><span class="symbol">c:</span> <span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30310</span>> <span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30430</span>></span><br><span class="line"><span class="number">4</span></span><br><span class="line"><span class="number">5</span></span><br><span class="line"><span class="number">6</span></span><br><span class="line"><span class="symbol">d:</span> <span class="params"><function main2.<locals></span>.<span class="params"><lambda></span> at <span class="number">0x10ca30310</span>> <span class="number">6</span></span><br></pre></td></tr></table></figure><h2 id="循环变量泄露"><a href="#循环变量泄露" class="headerlink" title="循环变量泄露"></a>循环变量泄露</h2><p>由于Python没有块级作用域，所以循环会改变当前作用域变量的值，也就是循环变量泄露。<br><strong>注意</strong>：Python3中列表推导式循环变量不会泄露，Python2中和常规循环一样泄露。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">x = <span class="number">-1</span></span><br><span class="line"><span class="keyword">for</span> x <span class="keyword">in</span> range(<span class="number">7</span>):</span><br><span class="line"> <span class="keyword">if</span> x == <span class="number">6</span>:</span><br><span class="line"> print(x, <span class="string">': for x inside loop'</span>)</span><br><span class="line">print(x, <span class="string">': x in global'</span>)</span><br></pre></td></tr></table></figure><p>输出如下</p><figure class="highlight basic"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">6 </span>: <span class="keyword">for</span> x inside loop</span><br><span class="line"><span class="symbol">6 </span>: x in global</span><br></pre></td></tr></table></figure><h2 id="闭包与延迟绑定"><a href="#闭包与延迟绑定" class="headerlink" title="闭包与延迟绑定"></a>闭包与延迟绑定</h2><p>再讲一下<strong>闭包</strong>，在一个内部函数中，对外部作用域的变量进行引用，(并且一般外部函数的返回值为内部函数)，那么内部函数就被认为是闭包。<br>这里所谓的引用可以也就是内部函数记住了<strong>变量的名称</strong>（而不是值，这个从ast语法树可以看出），而变量对应的值是会变化的。<br>如果在循环中定义闭包，引用的变量的值在循环结束才统一确定为最后一次循环时的值，也就是<strong>延迟绑定</strong>（lazy binding）。</p><p>所以下面的例子，<code>xx</code>的所有匿名函数的返回值均为<code>3</code><br><figure class="highlight maxima"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">xx = []</span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> [<span class="number">1</span>,<span class="number">2</span>,<span class="number">3</span>]:</span><br><span class="line"> xx.<span class="built_in">append</span>(<span class="built_in">lambda</span>: i)</span><br></pre></td></tr></table></figure></p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>再分析一开始的问题，这里的匿名函数引用了变量<code>i</code>，而<code>i</code>是全局变量，所以再次使用<code>i</code>作为循环变量时，列表中的匿名函数引用的值就被覆盖了。</p><p>正确做法：</p><ul><li>在独立的函数中定义闭包</li><li>闭包引用的变量应该是其他函数不可修改的</li><li>优先使用列表推导式</li></ul><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://stackoverflow.com/questions/3611760/scoping-in-python-for-loops" target="_blank" rel="noopener">https://stackoverflow.com/questions/3611760/scoping-in-python-for-loops</a></li><li><a href="https://www.educative.io/courses/python-ftw-under-the-hood/N8RW8508RkL" target="_blank" rel="noopener">https://www.educative.io/courses/python-ftw-under-the-hood/N8RW8508RkL</a></li><li><a href="https://mail.python.org/pipermail/python-ideas/2008-October/002109.html" target="_blank" rel="noopener">https://mail.python.org/pipermail/python-ideas/2008-October/002109.html</a></li></ul>]]></content>
<summary type="html">
<p>循环变量泄露与延迟绑定叠加在一起，会产生一些让人迷惑的结果。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Python" scheme="http://ponder.work/tags/Python/"/>
</entry>
<entry>
<title>bash 语法备忘</title>
<link href="http://ponder.work/2021/11/28/bash-cheat-sheet/"/>
<id>http://ponder.work/2021/11/28/bash-cheat-sheet/</id>
<published>2021-11-28T07:00:00.000Z</published>
<updated>2022-11-02T15:28:48.000Z</updated>
<content type="html"><![CDATA[<p>bash 语法作为程序员好像都了解一些，但又缺少体系化学习，需要使用到某些功能时又经常手忙脚乱地查。<br>本文主要参考<a href="https://wangdoc.com/bash/" target="_blank" rel="noopener">阮一峰的bash教程</a>，对bash的知识点进行了梳理。<br>本文目的是作为bash的语法备忘录、语法速查表。<br><a id="more"></a></p><h2 id="模式扩展"><a href="#模式扩展" class="headerlink" title="模式扩展"></a>模式扩展</h2><p>模式扩展（globbing），类似C语言中的宏展开，我们通常使用的通配符<code>*</code>就是其中之一。</p><p>Bash 一共提供八种扩展，前4种为文件扩展，只有文件路径确实存在才会扩展。</p><ul><li><code>~</code> 波浪线扩展</li><li><code>?</code> 问号扩展</li><li><code>*</code> 星号扩展</li><li><code>[]</code> 方括号扩展</li><li><code>{}</code> 大括号扩展</li><li><code>$var</code> 变量扩展</li><li><code>$(date)</code> 命令扩展</li><li><code>$((1 + 1))</code> 算术扩展</li></ul><h3 id="波浪线扩展"><a href="#波浪线扩展" class="headerlink" title="波浪线扩展"></a>波浪线扩展</h3><p>波浪线<code>~</code>会自动扩展成当前用户的主目录。<br><code>~user</code>表示扩展成用户<code>user</code>的主目录。如果用户不存在，则波浪号扩展不起作用。</p><figure class="highlight jboss-cli"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">bash-5.1$ <span class="keyword">echo</span> ~<span class="string">/projects/</span></span><br><span class="line"><span class="string">/Users/ruan/projects/</span></span><br><span class="line"></span><br><span class="line">bash-5.1$ <span class="keyword">echo</span> ~root/<span class="string">.ssh</span></span><br><span class="line"><span class="string">/var/root/.ssh</span></span><br><span class="line"></span><br><span class="line">bash-5.1$ <span class="keyword">echo</span> ~aaa/<span class="string">.ssh</span></span><br><span class="line">~aaa/<span class="string">.ssh</span></span><br></pre></td></tr></table></figure><h3 id="问号扩展"><a href="#问号扩展" class="headerlink" title="问号扩展"></a>问号扩展</h3><p><code>?</code>字符代表文件路径里面的任意单个字符，不包括空字符。<br><strong>只有文件确实存在的前提下，才会发生扩展。</strong></p><figure class="highlight armasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">bash-5.1$ </span>touch {a,<span class="keyword">b}.txt </span>ab.txt</span><br><span class="line"></span><br><span class="line"><span class="keyword">bash-5.1$ </span>ls ?.txt</span><br><span class="line"><span class="symbol">a.txt</span> <span class="keyword">b.txt</span></span><br><span class="line"><span class="keyword"></span></span><br><span class="line"><span class="keyword">bash-5.1$ </span>ls ??.txt</span><br><span class="line"><span class="symbol">ab.txt</span></span><br></pre></td></tr></table></figure><h3 id="星号扩展"><a href="#星号扩展" class="headerlink" title="星号扩展"></a>星号扩展</h3><p><code>*</code>字符代表文件路径里面的任意数量的任意字符，包括零个字符。</p><figure class="highlight dts"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ ls *.txt</span><br><span class="line">a.txt ab.txt b.txt</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ ls <span class="meta-keyword">/usr/</span>local/Cellar<span class="comment">/*/*/</span>bin/z*</span><br><span class="line"><span class="meta-keyword">/usr/</span>local/Cellar<span class="meta-keyword">/ffmpeg/</span><span class="number">4.4</span>_2<span class="meta-keyword">/bin/</span>zmqsend</span><br><span class="line"><span class="meta-keyword">/usr/</span>local/Cellar<span class="meta-keyword">/mysql-client/</span><span class="number">8.0</span><span class="number">.26</span><span class="meta-keyword">/bin/</span>zlib_decompress</span><br><span class="line"><span class="meta-keyword">/usr/</span>local/Cellar<span class="meta-keyword">/netpbm/</span><span class="number">10.86</span><span class="number">.24</span><span class="meta-keyword">/bin/</span>zeisstopnm</span><br><span class="line"><span class="meta-keyword">/usr/</span>local/Cellar<span class="meta-keyword">/perl/</span><span class="number">5.34</span><span class="number">.0</span><span class="meta-keyword">/bin/</span>zipdetails</span><br><span class="line"><span class="meta-keyword">/usr/</span>local/Cellar<span class="meta-keyword">/zstd/</span><span class="number">1.5</span><span class="number">.0</span><span class="meta-keyword">/bin/</span>zstd</span><br></pre></td></tr></table></figure><h3 id="方括号扩展"><a href="#方括号扩展" class="headerlink" title="方括号扩展"></a>方括号扩展</h3><p>方括号扩展的形式是<code>[...]</code>，只有文件确实存在的前提下才会扩展。</p><p><code>[^...]</code>和<code>[!...]</code>。它们表示匹配不在方括号里面的字符</p><p>方括号扩展有一个简写形式<code>[start-end]</code>，表示匹配一个连续的范围</p><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span>$ ls [ab].txt</span><br><span class="line"><span class="selector-tag">a</span><span class="selector-class">.txt</span> <span class="selector-tag">b</span>.txt</span><br><span class="line"></span><br><span class="line">bash-<span class="number">5.1</span>$ ls [^b]<span class="selector-tag">b</span>.txt</span><br><span class="line">ab.txt</span><br><span class="line"></span><br><span class="line">bash-<span class="number">5.1</span>$ ls [a-b].txt</span><br><span class="line"><span class="selector-tag">a</span><span class="selector-class">.txt</span> <span class="selector-tag">b</span>.txt</span><br></pre></td></tr></table></figure><h3 id="大括号扩展"><a href="#大括号扩展" class="headerlink" title="大括号扩展"></a>大括号扩展</h3><p>大括号扩展<code>{...}</code>表示分别扩展成大括号里面的所有值<br>大括号也可以与其他模式联用，并且总是先于其他模式进行扩展。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ echo {<span class="number">1</span>,<span class="number">2</span>,<span class="number">3</span>}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span></span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo a{<span class="number">1</span>,<span class="number">2</span>,<span class="number">3</span>}b</span><br><span class="line">a1b a2b a3b</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo --exclude={a,b,c}</span><br><span class="line">--exclude=a --exclude=b --exclude=c</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo foo{<span class="number">1</span>,<span class="number">2</span>{<span class="number">1</span>,<span class="number">2</span>}<span class="number">0</span>,<span class="number">3</span>}bar</span><br><span class="line">foo1bar foo210bar foo220bar foo3bar</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo {<span class="number">1.</span><span class="number">.3</span>}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span></span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo {<span class="number">1.</span><span class="number">.10</span>.<span class="number">.2</span>}</span><br><span class="line"><span class="number">1</span> <span class="number">3</span> <span class="number">5</span> <span class="number">7</span> <span class="number">9</span></span><br></pre></td></tr></table></figure><h3 id="变量扩展"><a href="#变量扩展" class="headerlink" title="变量扩展"></a>变量扩展</h3><p>Bash 将美元符号<code>$</code>开头的词元视为变量，将其扩展成变量值</p><figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span>$ echo <span class="variable">$HOME</span></span><br><span class="line"><span class="regexp">/Users/</span>ruan</span><br></pre></td></tr></table></figure><h3 id="命令扩展"><a href="#命令扩展" class="headerlink" title="命令扩展"></a>命令扩展</h3><p><code>$(...)</code>可以扩展成另一个命令的运行结果，该命令的所有输出都会作为返回值。</p><figure class="highlight dos"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5</span>.<span class="number">1</span>$ <span class="built_in">echo</span> $(<span class="built_in">date</span>)</span><br><span class="line">日 <span class="number">11</span> <span class="number">28</span> <span class="number">16</span>:<span class="number">22</span>:<span class="number">09</span> CST <span class="number">2021</span></span><br><span class="line"></span><br><span class="line">bash-<span class="number">5</span>.<span class="number">1</span>$ <span class="built_in">echo</span> `<span class="built_in">date</span>`</span><br><span class="line">日 <span class="number">11</span> <span class="number">28</span> <span class="number">16</span>:<span class="number">22</span>:<span class="number">24</span> CST <span class="number">2021</span></span><br></pre></td></tr></table></figure><h3 id="算术扩展"><a href="#算术扩展" class="headerlink" title="算术扩展"></a>算术扩展</h3><p><code>$((...))</code>可以扩展成整数运算的结果</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ echo $((<span class="number">1</span>+<span class="number">1</span>))</span><br><span class="line"><span class="number">2</span></span><br></pre></td></tr></table></figure><h2 id="引号使用"><a href="#引号使用" class="headerlink" title="引号使用"></a>引号使用</h2><h3 id="单引号"><a href="#单引号" class="headerlink" title="单引号"></a>单引号</h3><p>单引号用于保留字符的字面含义，在单引号里转义字符和模式扩展都会失效。</p><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span><span class="symbol">$</span> ls <span class="string">'[ab].txt'</span></span><br><span class="line">ls: cannot access <span class="string">'[ab].txt'</span>: <span class="keyword">No</span> such <span class="keyword">file</span> <span class="keyword">or</span> directory</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span><span class="symbol">$</span> ls <span class="string">'*'</span></span><br><span class="line">ls: cannot access <span class="string">'*'</span>: <span class="keyword">No</span> such <span class="keyword">file</span> <span class="keyword">or</span> directory</span><br></pre></td></tr></table></figure><h3 id="双引号"><a href="#双引号" class="headerlink" title="双引号"></a>双引号</h3><p>双引号比单引号宽松，三个特殊字符除外：美元符号（<code>$</code>）、反引号（<code>` </code>）和反斜杠（<code>\</code>）。这三个字符，会被 Bash 自动扩展。</p><p>也就是说，相比单引号在双引号中变量扩展，命令扩展，算术扩展以及转义字符是有效的。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">bash-5.1$ <span class="built_in">echo</span> <span class="string">"<span class="variable">$((1+1)</span>)"</span></span><br><span class="line">2</span><br><span class="line"></span><br><span class="line">bash-5.1$ <span class="built_in">echo</span> <span class="string">"<span class="variable">$HOME</span>"</span></span><br><span class="line">/Users/ruan</span><br><span class="line"></span><br><span class="line">bash-5.1$ <span class="built_in">echo</span> <span class="string">"<span class="variable">$(date)</span>"</span></span><br><span class="line">日 11 28 16:35:27 CST 2021</span><br><span class="line"></span><br><span class="line">bash-5.1$ <span class="built_in">echo</span> -e <span class="string">"1\t2"</span></span><br><span class="line">12</span><br></pre></td></tr></table></figure><h3 id="引号嵌套"><a href="#引号嵌套" class="headerlink" title="引号嵌套"></a>引号嵌套</h3><figure class="highlight nix"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 双引号中使用单引号</span></span><br><span class="line">bash-<span class="number">5.1</span>$ echo '<span class="string">"'</span></span><br><span class="line"><span class="string">"</span></span><br><span class="line">bash-<span class="number">5.1</span>$ echo '<span class="string">"<span class="subst">${HOME}</span>"</span>'</span><br><span class="line"><span class="string">"<span class="subst">${HOME}</span>"</span></span><br><span class="line"><span class="comment"># 单引号中使用双引号</span></span><br><span class="line">bash-<span class="number">5.1</span>$ echo <span class="string">"'<span class="subst">${HOME}</span>'"</span></span><br><span class="line">'/Users/ruan'</span><br><span class="line">bash-<span class="number">5.1</span>$ echo <span class="string">"'\"</span>${HOME}\<span class="string">"'"</span></span><br><span class="line">'<span class="string">"/Users/ruan"</span>'</span><br><span class="line"></span><br><span class="line"><span class="comment"># 引号嵌套中使用模式扩展，将需要扩展的字符放在单引号中；典型的有json变量填充</span></span><br><span class="line">bash-<span class="number">5.1</span>$ echo '{<span class="string">"user"</span>: <span class="string">"'<span class="subst">${USER}</span>'"</span>}'</span><br><span class="line">{<span class="string">"user"</span>: <span class="string">"ruan"</span>}</span><br></pre></td></tr></table></figure><h3 id="here-doc"><a href="#here-doc" class="headerlink" title="here doc"></a>here doc</h3><p>Here 文档（here document）是一种输入多行字符串的方法，格式如下。<br>它的格式分成开始标记（<code><< token</code>）和结束标记（<code>token</code>）, 一般用字符串<code>EOF</code>作为token</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><< token</span><br><span class="line">text</span><br><span class="line">token</span><br></pre></td></tr></table></figure><p>例如</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ cat << <span class="literal">EOF</span></span><br><span class="line">> <span class="number">11</span></span><br><span class="line">> <span class="number">22</span></span><br><span class="line">> <span class="number">33</span></span><br><span class="line">> <span class="literal">EOF</span></span><br><span class="line"><span class="number">11</span></span><br><span class="line"><span class="number">22</span></span><br><span class="line"><span class="number">33</span></span><br></pre></td></tr></table></figure><h3 id="here-string"><a href="#here-string" class="headerlink" title="here string"></a>here string</h3><p>Here 文档还有一个变体，叫做 Here 字符串（Here string），使用三个小于号（<code><<<</code>）表示。<br>它的作用是将字符串通过标准输入，传递给命令。</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span>$ <span class="keyword">cat</span> <<< foobar</span><br><span class="line">foobar</span><br><span class="line">bash-<span class="number">5.1</span>$ <span class="keyword">echo</span> foobar | <span class="keyword">cat</span></span><br><span class="line">foobar</span><br></pre></td></tr></table></figure><h2 id="变量"><a href="#变量" class="headerlink" title="变量"></a>变量</h2><p>bash 是基于标准输入在不同进程间交互数据的，大部分功能都是在操作字符串，所以<strong>变量的默认类型也是字符串</strong>。</p><h3 id="声明变量和读取变量"><a href="#声明变量和读取变量" class="headerlink" title="声明变量和读取变量"></a>声明变量和读取变量</h3><p>声明时等号两边不能有空格。<br>Bash 变量名区分大小写，<code>HOME</code>和<code>home</code>是两个不同的变量。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ foo=<span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $foo</span><br><span class="line"><span class="number">1</span></span><br></pre></td></tr></table></figure><h3 id="变量查看和删除"><a href="#变量查看和删除" class="headerlink" title="变量查看和删除"></a>变量查看和删除</h3><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 查看所有变量, 其中包含父进程export的变量</span></span><br><span class="line">bash-5.1$ set</span><br><span class="line"><span class="attribute">UID</span>=501</span><br><span class="line"><span class="attribute">USER</span>=ruan</span><br><span class="line"><span class="attribute">bar</span>=2</span><br><span class="line"><span class="attribute">foo</span>=1</span><br><span class="line"></span><br><span class="line">bash-5.1$ unset foo</span><br><span class="line">bash-5.1$ set</span><br><span class="line"><span class="attribute">UID</span>=501</span><br><span class="line"><span class="attribute">USER</span>=ruan</span><br><span class="line"><span class="attribute">bar</span>=2</span><br></pre></td></tr></table></figure><h3 id="变量输出"><a href="#变量输出" class="headerlink" title="变量输出"></a>变量输出</h3><p>用户创建的变量仅可用于当前 Shell，子 Shell 默认读取不到父 Shell 定义的变量。<br>如果希望子进程能够读到这个变量，需要使用export命令。</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">bash-5.1$ bash -c <span class="builtin-name">set</span> | grep foo</span><br><span class="line">bash-5.1$ <span class="builtin-name">export</span> <span class="attribute">foo</span>=1</span><br><span class="line">bash-5.1$ bash -c <span class="builtin-name">set</span> | grep foo</span><br><span class="line"><span class="attribute">foo</span>=1</span><br></pre></td></tr></table></figure><h3 id="环境变量"><a href="#环境变量" class="headerlink" title="环境变量"></a>环境变量</h3><p>平时所说的环境变量，就是init进程export输出的。子进程对变量的修改不会影响父进程，也就是说变量不是共享的。</p><figure class="highlight makefile"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 查看环境变量</span></span><br><span class="line">bash-5.1$ env</span><br><span class="line">SHELL=/bin/zsh</span><br><span class="line">LSCOLORS=Gxfxcxdxbxegedabagacad</span><br><span class="line">ITERM_PROFILE=Default</span><br></pre></td></tr></table></figure><p>下面是一些常见的环境变量。</p><ul><li><code>BASHPID</code>：Bash 进程的进程 ID。</li><li><code>BASHOPTS</code>：当前 Shell 的参数，可以用<code>shopt</code>命令修改。</li><li><code>DISPLAY</code>：图形环境的显示器名字，通常是<code>:0</code>，表示 X Server 的第一个显示器。</li><li><code>EDITOR</code>：默认的文本编辑器。</li><li><code>HOME</code>：用户的主目录。</li><li><code>HOST</code>：当前主机的名称。</li><li><code>IFS</code>：词与词之间的分隔符，默认为空格。</li><li><code>LANG</code>：字符集以及语言编码，比如<code>zh_CN.UTF-8</code>。</li><li><code>PATH</code>：由冒号分开的目录列表，当输入可执行程序名后，会搜索这个目录列表。</li><li><code>PS1</code>：Shell 提示符。</li><li><code>PS2</code>： 输入多行命令时，次要的 Shell 提示符。</li><li><code>PWD</code>：当前工作目录。</li><li><code>RANDOM</code>：返回一个0到32767之间的随机数。</li><li><code>SHELL</code>：Shell 的名字。</li><li><code>SHELLOPTS</code>：启动当前 Shell 的<code>set</code>命令的参数</li><li><code>TERM</code>：终端类型名，即终端仿真器所用的协议。</li><li><code>UID</code>：当前用户的 ID 编号。</li><li><code>USER</code>：当前用户的用户名。</li></ul><h3 id="特殊变量"><a href="#特殊变量" class="headerlink" title="特殊变量"></a>特殊变量</h3><p>Bash 提供一些特殊变量。这些变量的值由 Shell 提供，用户不能进行赋值。</p><ul><li><code>$?</code>: 上一个命令的退出码, 0为成功，其他为失败</li><li>$$$$: 当前进程的pid</li><li><code>$_</code>: 为上一个命令的最后一个参数</li><li><code>$!</code>: 为最近一个后台执行的异步命令的进程 ID。</li><li><code>$0</code>: bash脚本的参数列表，0是脚本文件路径，1到n是第1到第n个参数</li></ul><h3 id="变量默认值"><a href="#变量默认值" class="headerlink" title="变量默认值"></a>变量默认值</h3><ul><li><code>${varname:-word}</code>: 如果变量varname存在且不为空，则返回它的值，否则返回word</li><li><code>${varname:=word}</code>: 如果变量varname存在且不为空，则返回它的值，否则将它设为word，并且返回word。</li><li><code>${varname:+word}</code>: 如果变量名存在且不为空，则返回word，否则返回空值。它的目的是测试变量是否存在。</li><li><code>${varname:?message}</code>: 如果变量varname存在且不为空，则返回它的值，否则打印出varname: message，并中断脚本的执行。</li></ul><h3 id="declare-命令"><a href="#declare-命令" class="headerlink" title="declare 命令"></a>declare 命令</h3><p><code>declare</code>命令的主要参数（OPTION）如下。</p><ul><li><code>-a</code>：声明数组变量。</li><li><code>-A</code>：声明关联数组变量。</li><li><code>-f</code>：输出所有函数定义。</li><li><code>-F</code>：输出所有函数名。</li><li><code>-i</code>：声明整数变量。</li><li><code>-p</code>：查看变量信息。</li><li><code>-r</code>：声明只读变量。</li><li><code>-x</code>：该变量输出为环境变量。</li></ul><h2 id="数据类型"><a href="#数据类型" class="headerlink" title="数据类型"></a>数据类型</h2><p>bash 有字符串，数字，数字，关联数组四种数据类型，默认是字符串，其他类型需要手动声明。</p><h3 id="字符串"><a href="#字符串" class="headerlink" title="字符串"></a>字符串</h3><h4 id="定义"><a href="#定义" class="headerlink" title="定义"></a>定义</h4><p>语法 <code>varname=value</code></p><figure class="highlight armasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">bash-5.1$ </span><span class="built_in">s1</span><span class="symbol">=abcdefg</span></span><br><span class="line"><span class="keyword">bash-5.1$ </span>echo $<span class="built_in">s1</span></span><br><span class="line"><span class="symbol">abcdefg</span></span><br></pre></td></tr></table></figure><h4 id="获取长度（length）"><a href="#获取长度（length）" class="headerlink" title="获取长度（length）"></a>获取长度（length）</h4><p>语法 <code>${#varname}</code></p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">${</span><span class="comment">#s1}</span></span><br><span class="line"><span class="number">7</span></span><br></pre></td></tr></table></figure><h4 id="子字符串（substr）"><a href="#子字符串（substr）" class="headerlink" title="子字符串（substr）"></a>子字符串（substr）</h4><p>语法 <code>${varname:offset:length}</code>, offset为负数的时候，前面要加空格，防止与默认值语法冲突。</p><figure class="highlight armasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">bash-5.1$ </span><span class="built_in">s1</span><span class="symbol">=abcdefg</span></span><br><span class="line"><span class="keyword">bash-5.1$ </span>echo ${<span class="built_in">s1</span>:<span class="number">1</span>:<span class="number">3</span>}</span><br><span class="line"><span class="keyword">bcd</span></span><br><span class="line"><span class="keyword">bash-5.1$ </span>echo ${<span class="built_in">s1</span>: -<span class="number">6</span>:<span class="number">2</span>}</span><br><span class="line"><span class="keyword">bc</span></span><br><span class="line"><span class="keyword">bash-5.1$ </span>echo ${<span class="built_in">s1</span>: -<span class="number">6</span>:<span class="number">3</span>}</span><br><span class="line"><span class="keyword">bcd</span></span><br></pre></td></tr></table></figure><h4 id="替换-（replace）"><a href="#替换-（replace）" class="headerlink" title="替换 （replace）"></a>替换 （replace）</h4><h5 id="字符串头部的模式匹配"><a href="#字符串头部的模式匹配" class="headerlink" title="字符串头部的模式匹配"></a>字符串头部的模式匹配</h5><ul><li><code>${variable#pattern}</code>: 删除最短匹配（非贪婪匹配）的部分，返回剩余部分</li><li><code>${variable##pattern}</code>: 删除最长匹配（贪婪匹配）的部分，返回剩余部分</li></ul><p>匹配模式pattern可以使用<code>*</code>、<code>?</code>、<code>[]</code>等通配符。</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$ </span>myPath=<span class="regexp">/home/cam</span><span class="regexp">/book/long</span>.file.name</span><br><span class="line"></span><br><span class="line"><span class="variable">$ </span>echo <span class="variable">${</span>myPath<span class="comment">#/*/}</span></span><br><span class="line">cam/book/long.file.name</span><br><span class="line"></span><br><span class="line"><span class="variable">$ </span>echo <span class="variable">${</span>myPath<span class="comment">##/*/}</span></span><br><span class="line">long.file.name</span><br></pre></td></tr></table></figure><h5 id="字符串尾部的模式匹配"><a href="#字符串尾部的模式匹配" class="headerlink" title="字符串尾部的模式匹配"></a>字符串尾部的模式匹配</h5><ul><li><code>${variable%pattern}</code>: 删除最短匹配（非贪婪匹配）的部分，返回剩余部分</li><li><code>${variable%%pattern}</code>: 删除最长匹配（贪婪匹配）的部分，返回剩余部分</li></ul><figure class="highlight gradle"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ path=<span class="regexp">/home/</span>cam<span class="regexp">/book/</span><span class="keyword">long</span>.<span class="keyword">file</span>.name</span><br><span class="line"></span><br><span class="line">$ echo ${path%.*}</span><br><span class="line"><span class="regexp">/home/</span>cam<span class="regexp">/book/</span><span class="keyword">long</span>.<span class="keyword">file</span></span><br><span class="line"></span><br><span class="line">$ echo ${path%%.*}</span><br><span class="line"><span class="regexp">/home/</span>cam<span class="regexp">/book/</span><span class="keyword">long</span></span><br></pre></td></tr></table></figure><h5 id="任意位置的模式匹配"><a href="#任意位置的模式匹配" class="headerlink" title="任意位置的模式匹配"></a>任意位置的模式匹配</h5><p>如果匹配<code>pattern</code>则用<code>replace</code>替换匹配的内容</p><ul><li><code>${variable/pattern/replace}</code>: 替换第一个匹配</li><li><code>${variable//pattern/replace}</code>: 替换所有匹配</li></ul><figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ path=<span class="regexp">/home/</span>cam<span class="regexp">/foo/</span>foo.name</span><br><span class="line"></span><br><span class="line">$ echo <span class="variable">${path/foo/bar}</span></span><br><span class="line"><span class="regexp">/home/</span>cam<span class="regexp">/bar/</span>foo.name</span><br><span class="line"></span><br><span class="line">$ echo <span class="variable">${path//foo/bar}</span></span><br><span class="line"><span class="regexp">/home/</span>cam<span class="regexp">/bar/</span>bar.name</span><br></pre></td></tr></table></figure><h3 id="数字"><a href="#数字" class="headerlink" title="数字"></a>数字</h3><p>使用 <code>declare -i</code>声明整数变量。</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 声明为整数，可以直接计算，不需要使用$符号</span></span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>declare -i val1=<span class="number">12</span> val2=<span class="number">5</span></span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">$val1</span></span><br><span class="line"><span class="number">12</span></span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>val1+=val2</span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">$val1</span></span><br><span class="line"><span class="number">17</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 一个变量声明为整数以后，依然可以被改写为字符串。Bash 不会报错，但会赋以不确定的值</span></span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>val1=aaa</span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">$val1</span></span><br><span class="line"><span class="number">0</span></span><br></pre></td></tr></table></figure><h4 id="数值的进制"><a href="#数值的进制" class="headerlink" title="数值的进制"></a>数值的进制</h4><p>Bash 的数值默认都是十进制，但是在算术表达式中，也可以使用其他进制。</p><ul><li><code>number</code>：没有任何特殊表示法的数字是十进制数（以10为底）。</li><li><code>0number</code>：八进制数。</li><li><code>0xnumber</code>：十六进制数。</li><li><code>base#number</code>：<code>base</code>进制的数。</li></ul><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ declare -i a=<span class="number">0x77</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $a</span><br><span class="line"><span class="number">119</span></span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ declare -i a=<span class="number">0xfe</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $a</span><br><span class="line"><span class="number">254</span></span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ declare -i a=<span class="number">2</span>#<span class="number">111</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $a</span><br><span class="line"><span class="number">7</span></span><br></pre></td></tr></table></figure><h4 id="算术表达式"><a href="#算术表达式" class="headerlink" title="算术表达式"></a>算术表达式</h4><p><code>((...))</code>语法可以进行整数的算术运算。<br>支持的算术运算符如下。</p><ul><li><code>+</code>：加法</li><li><code>-</code>：减法</li><li><code>*</code>：乘法</li><li><code>/</code>：除法（整除）</li><li><code>%</code>：余数</li><li><code>**</code>：指数</li><li><code>++</code>：自增运算（前缀或后缀）</li><li><code>--</code>：自减运算（前缀或后缀）</li></ul><p>如果要读取算术运算的结果，需要在<code>((...))</code>前面加上美元符号<code>$((...))</code>，使其变成算术表达式，返回算术运算的值。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ echo $((<span class="number">1</span>+<span class="number">1</span>))</span><br><span class="line"><span class="number">2</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((<span class="number">1</span><span class="number">-1</span>))</span><br><span class="line"><span class="number">0</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((<span class="number">1</span>*<span class="number">2</span>))</span><br><span class="line"><span class="number">2</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((<span class="number">1</span>/<span class="number">2</span>))</span><br><span class="line"><span class="number">0</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((<span class="number">5</span>%<span class="number">2</span>))</span><br><span class="line"><span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ a=<span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((a++))</span><br><span class="line"><span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((a++))</span><br><span class="line"><span class="number">2</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $((++a))</span><br><span class="line"><span class="number">4</span></span><br></pre></td></tr></table></figure><h3 id="数组"><a href="#数组" class="headerlink" title="数组"></a>数组</h3><h4 id="创建数组"><a href="#创建数组" class="headerlink" title="创建数组"></a>创建数组</h4><p><code>array=(item1 item2)</code> 语法可初始化数组，括号内可以换行，多行初始化可以用<code>#</code>注释。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line"># 直接初始化数组</span><br><span class="line">bash<span class="number">-5.1</span>$ a=(<span class="number">1</span> <span class="number">2</span> <span class="number">3</span>)</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span></span><br><span class="line"></span><br><span class="line"># 多行初始化数组</span><br><span class="line">bash<span class="number">-5.1</span>$ a=(</span><br><span class="line">> <span class="number">1</span></span><br><span class="line">> <span class="number">2</span></span><br><span class="line">> <span class="number">3</span></span><br><span class="line">> #<span class="number">4</span></span><br><span class="line">> )</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span></span><br><span class="line"></span><br><span class="line"># 模式扩展初始化</span><br><span class="line">bash<span class="number">-5.1</span>$ a=({<span class="number">1.</span><span class="number">.3</span>})</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span></span><br><span class="line"></span><br><span class="line"># declare -a命令声明一个数组，也是可以的。</span><br><span class="line">bash<span class="number">-5.1</span>$ declare -a b</span><br><span class="line">bash<span class="number">-5.1</span>$ b+=(<span class="number">1</span>)</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${b[@]}</span><br><span class="line"><span class="number">1</span></span><br><span class="line"></span><br><span class="line">#read -a命令则是将用户的命令行输入，存入一个数组。</span><br><span class="line">bash<span class="number">-5.1</span>$ read -a c</span><br><span class="line"><span class="number">11</span> <span class="number">22</span> <span class="number">33</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${c[@]}</span><br><span class="line"><span class="number">11</span> <span class="number">22</span> <span class="number">33</span></span><br></pre></td></tr></table></figure><h4 id="访问数组元素"><a href="#访问数组元素" class="headerlink" title="访问数组元素"></a>访问数组元素</h4><p><code>array[index]</code> 语法可访问数组元素，不带index访问则是访问数组首个元素。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ a=(<span class="number">1</span> <span class="number">2</span> <span class="number">3</span>)</span><br><span class="line"># 查看元素</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">1</span>]}</span><br><span class="line"><span class="number">2</span></span><br><span class="line"># 元素赋值</span><br><span class="line">bash<span class="number">-5.1</span>$ a[<span class="number">1</span>]+=<span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">1</span>]}</span><br><span class="line"><span class="number">21</span></span><br><span class="line">bash<span class="number">-5.1</span>$ a[<span class="number">1</span>]=<span class="number">22</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">1</span>]}</span><br><span class="line"><span class="number">22</span></span><br><span class="line">#</span><br><span class="line">bash<span class="number">-5.1</span>$ a[<span class="number">0</span>]=<span class="number">1111</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a}</span><br><span class="line"><span class="number">1111</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">0</span>]}</span><br><span class="line"><span class="number">1111</span></span><br></pre></td></tr></table></figure><h4 id="数组长度"><a href="#数组长度" class="headerlink" title="数组长度"></a>数组长度</h4><p><code>${#array[@]}</code> 和 <code>${#array[*]}</code> 可访问获得数组长度</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>a=(<span class="number">1</span> <span class="number">2</span> <span class="number">3</span>)</span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">${</span><span class="comment">#a[*]}</span></span><br><span class="line"><span class="number">3</span></span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">${</span><span class="comment">#a[@]}</span></span><br><span class="line"><span class="number">3</span></span><br></pre></td></tr></table></figure><h4 id="获取非空元素下标"><a href="#获取非空元素下标" class="headerlink" title="获取非空元素下标"></a>获取非空元素下标</h4><p><code>${!array[@]}</code> 或 <code>${!array[*]}</code>, 可以获得非空元素的下标</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ a=(<span class="number">1</span> <span class="number">2</span> <span class="number">3</span>)</span><br><span class="line">bash<span class="number">-5.1</span>$ a[<span class="number">6</span>]=<span class="number">6</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">0</span>]}</span><br><span class="line"><span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span> <span class="number">6</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">3</span>]}</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[<span class="number">4</span>]}</span><br><span class="line"></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${!a[@]}</span><br><span class="line"><span class="number">0</span> <span class="number">1</span> <span class="number">2</span> <span class="number">6</span></span><br><span class="line"># 注意此时数组长度为<span class="number">4</span>，并不是<span class="number">7</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${#a[@]}</span><br><span class="line"><span class="number">4</span></span><br></pre></td></tr></table></figure><h4 id="数组元素切片"><a href="#数组元素切片" class="headerlink" title="数组元素切片"></a>数组元素切片</h4><p><code>${array[@]:position:length}</code>的语法可以提取数组成员。</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>a=({<span class="number">1</span>..<span class="number">10</span>})</span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">${</span>a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span> <span class="number">4</span> <span class="number">5</span> <span class="number">6</span> <span class="number">7</span> <span class="number">8</span> <span class="number">9</span> <span class="number">10</span></span><br><span class="line">bash-<span class="number">5.1</span><span class="variable">$ </span>echo <span class="variable">${</span>a[@]<span class="symbol">:</span><span class="number">1</span><span class="symbol">:</span><span class="number">3</span>}</span><br><span class="line"><span class="number">2</span> <span class="number">3</span> <span class="number">4</span></span><br></pre></td></tr></table></figure><h4 id="数组追加元素"><a href="#数组追加元素" class="headerlink" title="数组追加元素"></a>数组追加元素</h4><p>数组末尾追加元素，可以使用<code>+=</code>赋值运算符。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ a=({<span class="number">1.</span><span class="number">.10</span>})</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span> <span class="number">4</span> <span class="number">5</span> <span class="number">6</span> <span class="number">7</span> <span class="number">8</span> <span class="number">9</span> <span class="number">10</span></span><br><span class="line">bash<span class="number">-5.1</span>$ a+=(<span class="number">11</span> <span class="number">12</span>)</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">2</span> <span class="number">3</span> <span class="number">4</span> <span class="number">5</span> <span class="number">6</span> <span class="number">7</span> <span class="number">8</span> <span class="number">9</span> <span class="number">10</span> <span class="number">11</span> <span class="number">12</span></span><br></pre></td></tr></table></figure><h4 id="删除元素"><a href="#删除元素" class="headerlink" title="删除元素"></a>删除元素</h4><p>删除一个数组成员，使用<code>unset</code>命令。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ a=(<span class="number">1</span> <span class="number">2</span> <span class="number">3</span>)</span><br><span class="line"># 删除单个元素</span><br><span class="line">bash<span class="number">-5.1</span>$ unset a[<span class="number">1</span>]</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br><span class="line"><span class="number">1</span> <span class="number">3</span></span><br><span class="line"># 删除整个数组</span><br><span class="line">bash<span class="number">-5.1</span>$ unset a</span><br><span class="line">bash<span class="number">-5.1</span>$ echo ${a[@]}</span><br></pre></td></tr></table></figure><h3 id="关联数组"><a href="#关联数组" class="headerlink" title="关联数组"></a>关联数组</h3><p><code>declare -A</code>可以声明关联数组，关联数组使用字符串而不是整数作为数组索引。</p><p>除了初始化外，使用方法和数组基本相同</p><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">bash-<span class="number">5.1</span>$ declare -A a</span><br><span class="line">bash-<span class="number">5.1</span>$ <span class="selector-tag">a</span>[<span class="string">'red'</span>]=<span class="number">1</span></span><br><span class="line">bash-<span class="number">5.1</span>$ <span class="selector-tag">a</span>[<span class="string">'blue'</span>]=<span class="number">2</span></span><br><span class="line">bash-<span class="number">5.1</span>$ echo ${<span class="selector-tag">a</span>[@]}</span><br><span class="line"><span class="number">2</span> <span class="number">1</span></span><br><span class="line">bash-<span class="number">5.1</span>$ echo ${<span class="selector-tag">a</span>[@]:<span class="number">0</span>:<span class="number">1</span>}</span><br><span class="line"><span class="number">2</span></span><br></pre></td></tr></table></figure><h2 id="控制流"><a href="#控制流" class="headerlink" title="控制流"></a>控制流</h2><h3 id="注释"><a href="#注释" class="headerlink" title="注释"></a>注释</h3><p><code>#</code>表示注释，每行从<code>#</code>开始之后的内容代表注释，会被bash忽略.</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ echo <span class="number">1111</span> # <span class="number">222</span></span><br><span class="line"><span class="number">1111</span></span><br></pre></td></tr></table></figure><h3 id="条件判断"><a href="#条件判断" class="headerlink" title="条件判断"></a>条件判断</h3><p>bash 和常规编程语言一样使用<code>if</code>作为分支条件的关键字, <code>fi</code>作为结束的关键字，<code>else</code>和<br><code>elif</code>子句是可选的</p><p>其中<code>if</code>和<code>elif</code>的<code>condition</code>所判断的内容是命令的<a href="#状态码">状态码</a>是否为0，为0则执行关联的语句。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 因为bash中分号(;)和换行是等价的，所以有下面两种风格，其他多行语句也是类似的</span></span><br><span class="line"><span class="meta">#</span><span class="bash"> 本人偏好风格1</span></span><br><span class="line"><span class="meta">#</span><span class="bash"> 风格1</span></span><br><span class="line">if condition; then</span><br><span class="line"> command</span><br><span class="line">elif condition; then</span><br><span class="line"> command</span><br><span class="line">else</span><br><span class="line"> command</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 风格2</span></span><br><span class="line">if condition</span><br><span class="line">then</span><br><span class="line"> command</span><br><span class="line">elif condition</span><br><span class="line">then</span><br><span class="line"> command</span><br><span class="line">else</span><br><span class="line"> command</span><br><span class="line">fi</span><br></pre></td></tr></table></figure><p>这里的<code>condition</code>可以是多个命令，如<code>command1 && command2</code>，或者<code>command1 || command2</code>，则if判断的是这两个命令的状态码的逻辑计算结果。</p><p><code>condition</code>也是可以是<code>command1; command2</code>， 则则if判断的是最后一个命令的状态码。</p><p>这里最常用的<code>condition</code>是<code>test</code>命令, 也就是<code>[[]]</code>和<code>[]</code>. <code>test</code>是bash的内置命令，会执行给定的表达式，结果为真满足则返回状态码0, 否则返回状态码1.</p><p>下文循环语言的<code>condition</code>也是相同的，就不赘述了</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ test <span class="number">1</span> -eq <span class="number">1</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">bash<span class="number">-5.1</span>$ test <span class="number">1</span> -eq <span class="number">2</span></span><br><span class="line">bash<span class="number">-5.1</span>$ echo $?</span><br><span class="line"><span class="number">1</span></span><br></pre></td></tr></table></figure><p><code>[[]]</code>和<code>[]</code>的区别是<code>[[]]</code>内部支持<code>&&</code>，<code>||</code>逻辑判断，所以以下三种写法是等价的。</p><p>由于<code>[</code>和<code>]</code>是命令， 所以两侧一定要有空格，也是就是<code>[ 1 -eq 1 ]</code>，否则bash会认为命令找不到。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> <span class="built_in">test</span></span></span><br><span class="line">if test 1 -eq 2 || test 1 -eq 1; then</span><br><span class="line"> echo True</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> [ ]</span></span><br><span class="line">if [ 1 -eq 2 ] || [ 1 -eq 1 ]; then</span><br><span class="line"> echo True</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> [[ ]]</span></span><br><span class="line">if [[ 1 -eq 2 || 1 -eq 1 ]]; then</span><br><span class="line"> echo True</span><br><span class="line">fi</span><br></pre></td></tr></table></figure><h4 id="逻辑操作符"><a href="#逻辑操作符" class="headerlink" title="逻辑操作符"></a>逻辑操作符</h4><p>判断条件支持且（&&）或（||）非（!）</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> not </span></span><br><span class="line">if [[ ! 'aa' == 'bb' ]]; then</span><br><span class="line"> echo True</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> or </span></span><br><span class="line">if [[ 1 -eq 2 || 1 -eq 1 ]]; then</span><br><span class="line"> echo True</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> and</span></span><br><span class="line">if [[ 1 -ne 2 && 1 -eq 1 ]]; then</span><br><span class="line"> echo True</span><br><span class="line">fi</span><br></pre></td></tr></table></figure><h4 id="判断时引号使用（quote）"><a href="#判断时引号使用（quote）" class="headerlink" title="判断时引号使用（quote）"></a>判断时引号使用（quote）</h4><p>使用<code>[</code>和<code>test</code>时，变量引用注意加双引号，否则得不到正确的结果，<code>[[</code>则不需要。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">bash-3.2$ echo "$SSH_CLIENT"</span><br><span class="line"></span><br><span class="line">bash-3.2$ if [ -n $SSH_CLIENT ]; then echo 1; else echo 0; fi</span><br><span class="line">1</span><br><span class="line">bash-3.2$ if [ -n "$SSH_CLIENT" ]; then echo 1; else echo 0; fi</span><br><span class="line">0</span><br><span class="line">bash-3.2$ if [[ -n $SSH_CLIENT ]]; then echo 1; else echo 0; fi</span><br><span class="line">0</span><br></pre></td></tr></table></figure><h4 id="字符串判断"><a href="#字符串判断" class="headerlink" title="字符串判断"></a>字符串判断</h4><p>bash默认数据类型为字符串，所以常见的 <code>></code>, <code><</code>是用于字符串判断。</p><p>注意：字符串判断不支持<code>>=</code>和<code><=</code>, 得使用逻辑组合来替代</p><ul><li><code>-z string</code>：字符串串长度为0</li><li><code>-n string</code>: 字符串长度大于0</li><li><code>string1 == string2</code>: string1 等于 string2</li><li><code>string1 = string2</code>: string1 等于 string2</li><li><code>string1 > string2</code>: 如果按照字典顺序string1排列在string2之后</li><li><code>string1 < string2</code>: 如果按照字典顺序string1排列在string2之前</li></ul><h4 id="数字-整数-判断"><a href="#数字-整数-判断" class="headerlink" title="数字(整数)判断"></a>数字(整数)判断</h4><p>下面的表达式用于判断整数。</p><ul><li><code>[ integer1 -eq integer2 ]</code>：如果<code>integer1</code>等于<code>integer2</code>，则为<code>true</code>。</li><li><code>[ integer1 -ne integer2 ]</code>：如果<code>integer1</code>不等于<code>integer2</code>，则为<code>true</code>。</li><li><code>[ integer1 -le integer2 ]</code>：如果<code>integer1</code>小于或等于<code>integer2</code>，则为<code>true</code>。</li><li><code>[ integer1 -lt integer2 ]</code>：如果<code>integer1</code>小于<code>integer2</code>，则为<code>true</code>。</li><li><code>[ integer1 -ge integer2 ]</code>：如果<code>integer1</code>大于或等于<code>integer2</code>，则为<code>true</code>。</li><li><code>[ integer1 -gt integer2 ]</code>：如果<code>integer1</code>大于<code>integer2</code>，则为<code>true</code>。</li></ul><h4 id="文件判断"><a href="#文件判断" class="headerlink" title="文件判断"></a>文件判断</h4><p>以下表达式用来判断文件状态。仅列举常用判断，详细支持列表参考 <a href="https://tldp.org/LDP/abs/html/fto.html" target="_blank" rel="noopener">https://tldp.org/LDP/abs/html/fto.html</a></p><ul><li><code>[ -a file ]</code>：如果 file 存在，则为<code>true</code>。</li><li><code>[ -d file ]</code>：如果 file 存在并且是一个目录，则为<code>true</code>。</li><li><code>[ -e file ]</code>：如果 file 存在，则为<code>true</code>, 同<code>-a</code>。</li><li><code>[ -f file ]</code>：如果 file 存在并且是一个普通文件，则为<code>true</code>。</li><li><code>[ -h file ]</code>：如果 file 存在并且是符号链接，则为<code>true</code>。</li><li><code>[ -L file ]</code>：如果 file 存在并且是符号链接，则为<code>true</code>, 同<code>-h</code>。</li><li><code>[ -p file ]</code>：如果 file 存在并且是一个命名管道，则为<code>true</code>。</li><li><code>[ -r file ]</code>：如果 file 存在并且可读（当前用户有可读权限），则为<code>true</code>。</li><li><code>[ -s file ]</code>：如果 file 存在且其长度大于零，则为<code>true</code>。</li><li><code>[ -w file ]</code>：如果 file 存在并且可写（当前用户拥有可写权限），则为<code>true</code>。</li><li><code>[ -x file ]</code>：如果 file 存在并且可执行（有效用户有执行／搜索权限），则为<code>true</code>。</li></ul><h3 id="switch-case"><a href="#switch-case" class="headerlink" title="switch case"></a>switch case</h3><p>bash也支持，switch case，语法如下。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">case EXPRESSION in</span><br><span class="line"> PATTERN_1)</span><br><span class="line"> STATEMENTS</span><br><span class="line"> ;;</span><br><span class="line"> PATTERN_2)</span><br><span class="line"> STATEMENTS</span><br><span class="line"> ;;</span><br><span class="line"> PATTERN_N)</span><br><span class="line"> STATEMENTS</span><br><span class="line"> ;;</span><br><span class="line"> *)</span><br><span class="line"> STATEMENTS</span><br><span class="line"> ;;</span><br><span class="line">esac</span><br></pre></td></tr></table></figure><p>例如<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">a=2</span><br><span class="line"><span class="keyword">case</span> <span class="variable">$a</span> <span class="keyword">in</span></span><br><span class="line">1)</span><br><span class="line"> <span class="built_in">echo</span> 11</span><br><span class="line"> ;;</span><br><span class="line">2)</span><br><span class="line"> <span class="built_in">echo</span> 22</span><br><span class="line"> ;;</span><br><span class="line">*)</span><br><span class="line"> ;;</span><br><span class="line"><span class="keyword">esac</span></span><br></pre></td></tr></table></figure></p><h3 id="循环"><a href="#循环" class="headerlink" title="循环"></a>循环</h3><h4 id="while-循环"><a href="#while-循环" class="headerlink" title="while 循环"></a>while 循环</h4><p><code>while</code>循环有一个判断条件，只要符合条件，就不断循环执行指定的语句。<br><code>condition</code>与if语句的相同，就不赘述了。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">while condition; do</span><br><span class="line"> command</span><br><span class="line">done</span><br></pre></td></tr></table></figure><h4 id="unitl-循环"><a href="#unitl-循环" class="headerlink" title="unitl 循环"></a>unitl 循环</h4><p><code>until</code>循环与<code>while</code>循环恰好相反，只要不符合判断条件（判断条件失败），就不断循环执行指定的语句。一旦符合判断条件，就退出循环。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">until condition; do</span><br><span class="line"> command</span><br><span class="line">done</span><br></pre></td></tr></table></figure><h4 id="for-in-循环"><a href="#for-in-循环" class="headerlink" title="for-in 循环"></a>for-in 循环</h4><p><code>for...in</code>循环用于遍历列表的每一项。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">for variable in list; do</span><br><span class="line"> commands</span><br><span class="line">done</span><br></pre></td></tr></table></figure><p>常见的几种用法<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">for i in 1 2 3; do</span><br><span class="line"> echo $i</span><br><span class="line">done</span><br><span class="line"></span><br><span class="line">for i in {1..3}; do</span><br><span class="line"> echo $i</span><br><span class="line">done</span><br><span class="line"></span><br><span class="line">list=(1 2 3)</span><br><span class="line">for i in ${list[@]}; do</span><br><span class="line"> echo $i</span><br><span class="line">done</span><br></pre></td></tr></table></figure></p><h4 id="for-循环"><a href="#for-循环" class="headerlink" title="for 循环"></a>for 循环</h4><p><code>for</code>循环还支持 C 语言的循环语法。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">for (( expression1; expression2; expression3 )); do</span><br><span class="line"> commands</span><br><span class="line">done</span><br></pre></td></tr></table></figure><p>上面代码中，<code>expression1</code>用来初始化循环条件，<code>expression2</code>用来决定循环结束的条件，<code>expression3</code>在每次循环迭代的末尾执行，用于更新值。</p><p>注意，循环条件放在双重圆括号之中。另外，圆括号之中使用变量，不必加上美元符号<code>$</code>。</p><p>例如<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">for ((i=1; i<=3; i++)); do</span><br><span class="line"> echo $i</span><br><span class="line">done</span><br></pre></td></tr></table></figure></p><h4 id="跳出循环"><a href="#跳出循环" class="headerlink" title="跳出循环"></a>跳出循环</h4><p>Bash 提供了两个内部命令<code>break</code>和<code>continue</code>，用来在循环内部跳出循环。</p><p><code>break</code>命令立即终止循环，程序继续执行循环块之后的语句，即不再执行剩下的循环。<br><code>continue</code>命令立即终止本轮循环，开始执行下一轮循环。</p><h2 id="函数"><a href="#函数" class="headerlink" title="函数"></a>函数</h2><h3 id="函数定义"><a href="#函数定义" class="headerlink" title="函数定义"></a>函数定义</h3><p>Bash 函数定义的语法有两种，其中<code>fn</code>为定义的函数名称。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 第一种</span></span><br><span class="line">fn() {</span><br><span class="line"><span class="meta"> #</span><span class="bash"> codes</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 第二种</span></span><br><span class="line">function fn() {</span><br><span class="line"><span class="meta"> #</span><span class="bash"> codes</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="函数参数"><a href="#函数参数" class="headerlink" title="函数参数"></a>函数参数</h3><p>函数体内可以使用参数变量，获取函数参数。函数的参数变量，与脚本参数变量是一致的。</p><ul><li><code>${N}</code>：函数的第一个到第N个的参数。</li><li><code>$0</code>：函数所在的脚本名。</li><li><code>$#</code>：函数的参数总数。</li><li><code>$@</code>：函数的全部参数，参数之间使用空格分隔。</li><li><code>$*</code>：函数的全部参数，参数之间使用变量<code>$IFS</code>值的第一个字符分隔，默认为空格，但是可以自定义。</li></ul><h3 id="函数调用"><a href="#函数调用" class="headerlink" title="函数调用"></a>函数调用</h3><p><code>funcname arg1 arg ... argN</code> 的语法进行函数调用。主要函数的返回值和输出值（标准输出）的区别，这和主流编程语言不同</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">add() {</span><br><span class="line"> declare -i res</span><br><span class="line"> res=0</span><br><span class="line"> for i in $@; do</span><br><span class="line"> res+=$i</span><br><span class="line"> done</span><br><span class="line"> echo $res</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 结果为10</span></span><br><span class="line">add 1 2 3 4</span><br></pre></td></tr></table></figure><h3 id="函数返回值"><a href="#函数返回值" class="headerlink" title="函数返回值"></a>函数返回值</h3><p><code>return</code>命令用于从函数返回一个值。返回值和命令的状态码一样，可以用<code>$?</code>拿到值。<br><code>return</code>也可以不接具体的值，则返回值是return命令的上一条命令的状态码。<br>如果不加<code>return</code>，则返回值是函数体最后一条命令的状态码。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">function func_return_value {</span><br><span class="line"> return 10</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="关键概念"><a href="#关键概念" class="headerlink" title="关键概念"></a>关键概念</h2><h3 id="shebang"><a href="#shebang" class="headerlink" title="shebang"></a>shebang</h3><p>Shebang（也称为Hashbang）是一个由井号和叹号构成的字符序列<code>#!</code>， 其出现在可执行文本文件的第一行的前两个字符。<br>在文件中存在Shebang的情况下，类Unix操作系统的程序加载器会分析Shebang后的内容，将这些内容作为解释器指令，并调用该指令.</p><p>例如，shell脚本<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line"></span><br><span class="line"><span class="built_in">echo</span> Hello, world!</span><br></pre></td></tr></table></figure></p><p>python 脚本<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#!/usr/bin/env python -u</span></span><br><span class="line"></span><br><span class="line">print(<span class="string">"Hello, world!"</span>)</span><br></pre></td></tr></table></figure></p><h3 id="状态码"><a href="#状态码" class="headerlink" title="状态码"></a>状态码</h3><p>每个命令都会返回一个退出状态码（有时候也被称为返回状态）。</p><p>成功的命令返回 0，不成功的命令返回非零值，非零值通常都被解释成一个错误码。行为良好的 UNIX 命令、程序和工具都会返回 0 作为退出码来表示成功，虽然偶尔也会有例外。</p><p>状态码一般是程序的main函数的返回码，如c,c++。<br>如果是bash脚本，状态码的值则是 <code>exit</code> 命令的参数值。<br>当脚本以不带参数的 <code>exit</code> 命令来结束时，脚本的退出状态码就由脚本中最后执行的命令来决定，这与函数的 <code>return</code> 行为是一致的。</p><p>特殊变量<code>$?</code>可以查看上个命令的退出状态码</p><h3 id="文件描述符"><a href="#文件描述符" class="headerlink" title="文件描述符"></a>文件描述符</h3><p>文件描述符在形式上是一个非负整数。指向内核为每一个进程所维护的该进程打开文件的记录表。<br>当程序打开一个现有文件或者创建一个新文件时，内核向进程返回一个文件描述符。</p><h4 id="标准输入输出"><a href="#标准输入输出" class="headerlink" title="标准输入输出"></a>标准输入输出</h4><p>每个Unix进程（除了可能的守护进程）应均有三个标准的POSIX文件描述符，对应于三个标准流：</p><ul><li><code>0</code>：标准输入</li><li><code>1</code>：标准输出</li><li><code>2</code>：错误输出</li></ul><h4 id="打开新的文件描述符"><a href="#打开新的文件描述符" class="headerlink" title="打开新的文件描述符"></a>打开新的文件描述符</h4><p>手动指定描述符<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">exec 3<> /tmp/foo #open fd 3.</span><br><span class="line">echo "test" >&3</span><br><span class="line">exec 3>&- #close fd 3.</span><br></pre></td></tr></table></figure></p><p>系统自动分配描述符，bash4.1开始支持(在macos报错，原因不明)<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash">!/bin/bash</span></span><br><span class="line"></span><br><span class="line">FILENAME=abc.txt</span><br><span class="line"></span><br><span class="line">exec {FD}<>"$FILENAME"</span><br><span class="line">echo 11 >&FD</span><br><span class="line">echo 22 >&FD</span><br><span class="line"><span class="meta">$</span><span class="bash">FD>&-</span></span><br></pre></td></tr></table></figure></p><h4 id="描述符重定向"><a href="#描述符重定向" class="headerlink" title="描述符重定向"></a>描述符重定向</h4><ul><li><code>command > file</code>: 将输出重定向到 file。</li><li><code>command < file</code>: 将输入重定向到 file。</li><li><code>command >> file</code>: 将输出以追加的方式重定向到 file。</li><li><code>n > file</code>: 将文件描述符为 n 的文件重定向到 file。</li><li><code>n >> file</code>: 将文件描述符为 n 的文件以追加的方式重定向到 file。</li><li><code>n >& m</code>: 将输出文件 m 和 n 合并。</li><li><code>n <& m</code>: 将输入文件 m 和 n 合并。</li></ul><p>所以命令中常见的<code>ls -al > output.txt 2>&1</code>, 就是将标准输出和错误输出都重定向到一个文件。<br>等价于<code>ls -al &>output.txt</code>，本人偏好这种写法，比较简洁。</p><h3 id="IFS-Input-Field-Separators"><a href="#IFS-Input-Field-Separators" class="headerlink" title="IFS (Input Field Separators)"></a>IFS (Input Field Separators)</h3><p>IFS决定了bash在处理字符串的时候是如何进行单词切分。<br>IFS的默认值是空格，TAB，换行符，即<code>\t\n</code></p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">echo</span> <span class="string">"<span class="variable">$IFS</span>"</span> | cat -et</span></span><br><span class="line"> ^I$</span><br><span class="line"><span class="meta">$</span><span class="bash"></span></span><br></pre></td></tr></table></figure><p>例如，在for循环的时候，如何区分每个item<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">for i in `echo -e "foo bar\tfoobar\nfoofoo"`; do</span><br><span class="line"> echo "'$i' is the substring";</span><br><span class="line">done</span><br></pre></td></tr></table></figure></p><p>也可以自定义<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">OLD_IFS="$IFS"</span><br><span class="line">IFS=":"</span><br><span class="line">string="1:2:3"</span><br><span class="line">for i in $string; do</span><br><span class="line"> echo "'$i' is the substring";</span><br><span class="line">done</span><br><span class="line">IFS=$OLD_IFS</span><br></pre></td></tr></table></figure></p><h3 id="任务管理"><a href="#任务管理" class="headerlink" title="任务管理"></a>任务管理</h3><p>linux进程分前台（fg）和后台（bg）。</p><p>在命令的末尾添加<code>&</code>可以将命令后台执行，一般配合输出重定向使用。</p><p><code>jobs</code>可以查看当前bash进程的子进程，并通过<code>fg</code>和<code>bg</code>进行前台和后台切换。</p><p><code>%1</code>代表后台的第一个进程，以此类推<code>%N</code>代表第n个.</p><p><code>control + Z</code>可以将当前前台程序暂停，配合<code>bg</code>可以将其转后台。</p><p><code>wait [pid]</code>可以等待子进程结束，如果不带pid参数则等待所有子进程结束。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">bash<span class="number">-5.1</span>$ sleep <span class="number">100</span></span><br><span class="line">^Z</span><br><span class="line">[<span class="number">1</span>]+ 已停止 sleep <span class="number">100</span></span><br><span class="line">bash<span class="number">-5.1</span>$ jobs</span><br><span class="line">[<span class="number">1</span>]+ 已停止 sleep <span class="number">100</span></span><br><span class="line">bash<span class="number">-5.1</span>$ bg %<span class="number">1</span></span><br><span class="line">[<span class="number">1</span>]+ sleep <span class="number">100</span> &</span><br><span class="line">bash<span class="number">-5.1</span>$ sleep <span class="number">200</span> &</span><br><span class="line">[<span class="number">2</span>] <span class="number">63603</span></span><br><span class="line">bash<span class="number">-5.1</span>$ jobs</span><br><span class="line">[<span class="number">1</span>]- 运行中 sleep <span class="number">100</span> &</span><br><span class="line">[<span class="number">2</span>]+ 运行中 sleep <span class="number">200</span> &</span><br><span class="line">bash<span class="number">-5.1</span>$ wait %<span class="number">1</span></span><br><span class="line">[<span class="number">1</span>]- 已完成 sleep <span class="number">100</span></span><br></pre></td></tr></table></figure><h4 id="后台进程并发控制"><a href="#后台进程并发控制" class="headerlink" title="后台进程并发控制"></a>后台进程并发控制</h4><p>可以利用jobs对后台进程并发数目进行控制<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">for i in {1..30}; do</span><br><span class="line">sleep $((30+i)) &</span><br><span class="line">if [[ $(jobs | wc -l ) -gt 10 ]]; then</span><br><span class="line">jobs</span><br><span class="line">wait</span><br><span class="line">fi</span><br><span class="line">done</span><br><span class="line">wait</span><br></pre></td></tr></table></figure></p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://wangdoc.com/bash/" target="_blank" rel="noopener">https://wangdoc.com/bash/</a></li><li><a href="https://tldp.org/LDP/abs/html/fto.html" target="_blank" rel="noopener">https://tldp.org/LDP/abs/html/fto.html</a></li><li><a href="https://zh.wikipedia.org/wiki/Shebang" target="_blank" rel="noopener">https://zh.wikipedia.org/wiki/Shebang</a></li><li><a href="https://en.wikipedia.org/wiki/File_descriptor" target="_blank" rel="noopener">https://en.wikipedia.org/wiki/File_descriptor</a></li></ul>]]></content>
<summary type="html">
<p>bash 语法作为程序员好像都了解一些，但又缺少体系化学习，需要使用到某些功能时又经常手忙脚乱地查。<br>本文主要参考<a href="https://wangdoc.com/bash/" target="_blank" rel="noopener">阮一峰的bash教程</a>，对bash的知识点进行了梳理。<br>本文目的是作为bash的语法备忘录、语法速查表。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Shell" scheme="http://ponder.work/tags/Shell/"/>
</entry>
<entry>
<title>MySQL 自定义数据库路径</title>
<link href="http://ponder.work/2021/10/14/mysql-custom-location/"/>
<id>http://ponder.work/2021/10/14/mysql-custom-location/</id>
<published>2021-10-14T12:17:00.000Z</published>
<updated>2021-10-14T16:05:34.000Z</updated>
<content type="html"><![CDATA[<p><em>最近的一些文章是整理以前的笔记</em><br>MySQL 是最常用的数据，有时希望将数据库文件存放在自定义路径，或者在系统中启动多个 MySQL服务。<br><a id="more"></a><br>当然，如果条件允许，建议直接使用 docker </p><h2 id="创建-my-cnf-配置文件"><a href="#创建-my-cnf-配置文件" class="headerlink" title="创建 my.cnf 配置文件"></a>创建 my.cnf 配置文件</h2><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="section">[mysqld]</span></span><br><span class="line"><span class="attr">datadir</span>=/home/ruan/data/mysql_data</span><br><span class="line"><span class="attr">socket</span>=/home/ruan/data/mysql.sock</span><br><span class="line"><span class="attr">user</span>=ruan</span><br><span class="line"><span class="comment"># Disabling symbolic-links is recommended to prevent assorted security risks</span></span><br><span class="line"><span class="attr">symbolic-links</span>=<span class="number">0</span></span><br><span class="line"><span class="attr">bind-address</span>=<span class="number">127.0</span>.<span class="number">0.1</span></span><br><span class="line"><span class="attr">port</span> = <span class="number">12345</span></span><br><span class="line"></span><br><span class="line"><span class="attr">character-set-server</span>=utf8</span><br><span class="line"><span class="attr">collation-server</span>=utf8_general_ci</span><br><span class="line"></span><br><span class="line"><span class="section">[mysqld_safe]</span></span><br><span class="line"><span class="attr">log-error</span>=/home/ruan/data/mysqld.log</span><br><span class="line"><span class="attr">pid-file</span>=/home/ruan/data/mysqld.pid</span><br></pre></td></tr></table></figure><h2 id="启动和初始化"><a href="#启动和初始化" class="headerlink" title="启动和初始化"></a>启动和初始化</h2><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 启动MySQL</span></span><br><span class="line">mysqld_safe --defaults-file=my.cnf --user=ruan</span><br><span class="line"><span class="meta">#</span><span class="bash"> 初始化数据库</span></span><br><span class="line">mysql_install_db --defaults-file=my.cnf --user=ruan</span><br></pre></td></tr></table></figure><h2 id="目录结构"><a href="#目录结构" class="headerlink" title="目录结构"></a>目录结构</h2><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">$ tree data</span><br><span class="line">data</span><br><span class="line">├── my.cnf</span><br><span class="line">├── mysql_data</span><br><span class="line">│ ├── ibdata1</span><br><span class="line">│ ├── ib_logfile0</span><br><span class="line">│ ├── ib_logfile1</span><br><span class="line">├── mysqld.log</span><br><span class="line">├── mysqld.pid</span><br><span class="line">└── mysql.sock</span><br></pre></td></tr></table></figure><h2 id="设置密码"><a href="#设置密码" class="headerlink" title="设置密码"></a>设置密码</h2><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">mysql> use mysql; update<span class="built_in"> user </span><span class="builtin-name">set</span> <span class="attribute">password</span>=password('m654321') where <span class="attribute">user</span>=<span class="string">'root'</span>; flush privileges;</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p><em>最近的一些文章是整理以前的笔记</em><br>MySQL 是最常用的数据，有时希望将数据库文件存放在自定义路径，或者在系统中启动多个 MySQL服务。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="MySQL" scheme="http://ponder.work/tags/MySQL/"/>
</entry>
<entry>
<title>hexo 站内搜索内容不完全问题修复</title>
<link href="http://ponder.work/2021/10/11/hexo-local-search-not-complete-fix/"/>
<id>http://ponder.work/2021/10/11/hexo-local-search-not-complete-fix/</id>
<published>2021-10-11T12:00:00.000Z</published>
<updated>2024-07-09T13:50:24.831Z</updated>
<content type="html"><![CDATA[<p>在使用 Hexo 的站内搜索时，发现搜索的内容不全。单步调试发现xml解析不完整，有部分内容被截断了。</p><p>在浏览器中打开<a href="/search.xml">/search.xml</a>发现以下错误。显然xml中有非法字符，xml解析产生了错误。<br><a id="more"></a><br><img src="https://image.ponder.work/mweb/2021-10-12-16339770374988.jpg" alt></p><p>将search.xml文件保存，并用python打开，找到具体出错的位置。<br>utf8解码之后可以发现<code>\x10</code>非法字符，将其删除，重新生成文章问题解决。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">>>> </span>xxx = open(<span class="string">'./tmp.xml'</span>, <span class="string">'rb'</span>).read()</span><br><span class="line"><span class="meta">>>> </span>xxx.index(<span class="string">b'\x10\xE7\x84\xB6'</span>)</span><br><span class="line"><span class="number">923278</span></span><br><span class="line"><span class="meta">>>> </span>xxx[<span class="number">923278</span>:<span class="number">923278</span>+<span class="number">31</span>]</span><br><span class="line"><span class="string">b'\x10\xe7\x84\xb6\xe5\x88\x99\xef\xbc\x8c\xe4\xbb\x8a\xe4\xb9\x8b\xe4\xb8\x96\xe4\xba\xba\xef\xbc\x8c\xe7\xb1\xbb\xe6\xad\xa4'</span></span><br><span class="line"><span class="meta">>>> </span>xxx[<span class="number">923278</span>:<span class="number">923278</span>+<span class="number">31</span>].decode()</span><br><span class="line"><span class="string">'\x10然则，今之世人，类此'</span></span><br></pre></td></tr></table></figure><p><img src="https://image.ponder.work/mweb/2021-10-12-16339793845815.jpg" alt></p>]]></content>
<summary type="html">
<p>在使用 Hexo 的站内搜索时，发现搜索的内容不全。单步调试发现xml解析不完整，有部分内容被截断了。</p>
<p>在浏览器中打开<a href="/search.xml">/search.xml</a>发现以下错误。显然xml中有非法字符，xml解析产生了错误。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="XML" scheme="http://ponder.work/tags/XML/"/>
</entry>
<entry>
<title>MySQL 表分区使用</title>
<link href="http://ponder.work/2021/10/10/mysql-partition/"/>
<id>http://ponder.work/2021/10/10/mysql-partition/</id>
<published>2021-10-10T07:00:00.000Z</published>
<updated>2021-10-11T18:13:59.000Z</updated>
<content type="html"><![CDATA[<p>使用MySQL数据库时，当表的数据条数比较大时（1000w以上），数据查询会很慢，索引的效果也不好。</p><p>这时我们可以把表的数据分区存储，安装数据值的前缀或者时间字段来分区。<br><a id="more"></a></p><h2 id="建表"><a href="#建表" class="headerlink" title="建表"></a>建表</h2><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> test_part (</span><br><span class="line"> appid <span class="built_in">int</span>(<span class="number">11</span>),</span><br><span class="line"> val <span class="built_in">int</span>(<span class="number">11</span>),</span><br><span class="line"> username <span class="built_in">VARCHAR</span>(<span class="number">25</span>) <span class="keyword">NOT</span> <span class="literal">NULL</span>,</span><br><span class="line"> start_time DATETIME</span><br><span class="line">)</span><br><span class="line"><span class="keyword">PARTITION</span> <span class="keyword">BY</span> <span class="keyword">RANGE</span> (<span class="keyword">TO_DAYS</span>(start_time) )(</span><br><span class="line"> <span class="keyword">PARTITION</span> p20190305 <span class="keyword">VALUES</span> <span class="keyword">LESS</span> <span class="keyword">THAN</span> (<span class="keyword">TO_DAYS</span>(<span class="string">'2019-03-06 00:00:00'</span>) )</span><br><span class="line">)</span><br></pre></td></tr></table></figure><h2 id="删除分区"><a href="#删除分区" class="headerlink" title="删除分区"></a>删除分区</h2><p>alter table test_part drop partition p1;</p><p>不可以删除hash或者key分区。</p><p>一次性删除多个分区，alter table test_part drop partition p1,p2;</p><h2 id="增加分区"><a href="#增加分区" class="headerlink" title="增加分区"></a>增加分区</h2><p>ALTER TABLE test_part ADD partition (partition p20190306 VALUES LESS THAN (TO_DAYS(‘2019-03-07 00:00:00’))); </p><h2 id="生成测试数据"><a href="#生成测试数据" class="headerlink" title="生成测试数据"></a>生成测试数据</h2><p>创建储存过程<br><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">DROP</span> <span class="keyword">PROCEDURE</span> <span class="keyword">IF</span> <span class="keyword">EXISTS</span> proc1;</span><br><span class="line">DELIMITER $$</span><br><span class="line"><span class="keyword">SET</span> AUTOCOMMIT = <span class="number">0</span>$$</span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">PROCEDURE</span> proc1()</span><br><span class="line"><span class="keyword">BEGIN</span></span><br><span class="line"><span class="keyword">DECLARE</span> v_cnt <span class="built_in">DECIMAL</span> (<span class="number">10</span>) <span class="keyword">DEFAULT</span> <span class="number">0</span> ;</span><br><span class="line">dd:LOOP</span><br><span class="line"> <span class="keyword">INSERT</span> <span class="keyword">INTO</span> test_part <span class="keyword">VALUES</span> (</span><br><span class="line"> <span class="keyword">FLOOR</span>(<span class="keyword">RAND</span>()*<span class="number">100</span>), </span><br><span class="line"> <span class="keyword">FLOOR</span>(<span class="keyword">RAND</span>()*<span class="number">1000</span>), </span><br><span class="line"> <span class="keyword">UUID</span>(), </span><br><span class="line"> <span class="keyword">DATE_ADD</span>(<span class="string">'2019-03-04 00:00:00'</span>, <span class="built_in">INTERVAL</span> <span class="keyword">FLOOR</span>(v_cnt / <span class="number">5000</span>) <span class="keyword">MINUTE</span>)</span><br><span class="line"> );</span><br><span class="line"> <span class="keyword">COMMIT</span>;</span><br><span class="line"> <span class="keyword">SET</span> v_cnt = v_cnt+<span class="number">1</span> ;</span><br><span class="line"> IF v_cnt = 10000000 THEN LEAVE dd;</span><br><span class="line"> <span class="keyword">END</span> <span class="keyword">IF</span>;</span><br><span class="line"> <span class="keyword">END</span> <span class="keyword">LOOP</span> dd ;</span><br><span class="line"><span class="keyword">END</span>;$$</span><br><span class="line">DELIMITER ;</span><br><span class="line">``` </span><br><span class="line"></span><br><span class="line">调用储存过程</span><br></pre></td></tr></table></figure></p><p> call proc1;<br>```</p>]]></content>
<summary type="html">
<p>使用MySQL数据库时，当表的数据条数比较大时（1000w以上），数据查询会很慢，索引的效果也不好。</p>
<p>这时我们可以把表的数据分区存储，安装数据值的前缀或者时间字段来分区。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="MySQL" scheme="http://ponder.work/tags/MySQL/"/>
</entry>
<entry>
<title>macOS 使用技巧</title>
<link href="http://ponder.work/2021/10/10/macos-tips/"/>
<id>http://ponder.work/2021/10/10/macos-tips/</id>
<published>2021-10-10T07:00:00.000Z</published>
<updated>2024-07-09T13:50:24.797Z</updated>
<content type="html"><![CDATA[<p>对程序员来说，macOS 就是一个桌面支持比较好的 Linux/Unix，给日常开发带来了许多便利</p><p>本文记录一些日常使用中的小技巧。<br><a id="more"></a></p><h2 id="macOS-mojave-密码位数限制"><a href="#macOS-mojave-密码位数限制" class="headerlink" title="macOS mojave 密码位数限制"></a>macOS mojave 密码位数限制</h2><p>苹果在10.14的系统上增加了密码最小长度的限制<br>可以通过这个命令去除</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pwpolicy -clearaccountpolicies</span><br></pre></td></tr></table></figure><p><strong>注意</strong>：去除限制之后，通过time machine备份的系统，还原到有限制的机器上，可能会导致一些bug。如卡死在登陆页面，部分应用的资源库损坏等等。</p><h2 id="重置账户、配置"><a href="#重置账户、配置" class="headerlink" title="重置账户、配置"></a>重置账户、配置</h2><p>如果出现上面的问题，可以尝试重置下系统设置（选择任意一种，建议选1或2）</p><ol><li>新建管理员账号，在新机器上去除密码限制。</li><li><p>进入恢复模式（开机按command+r），删除对应用户账户配置文件（恢复模式下文件挂载路径可能不同）, 重新开机</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">mv ~/Library/Preferences ~/Library/Preferences.bak</span><br><span class="line">mv ~/Library/PreferencePanes ~/Library/PreferencePanes.bak</span><br></pre></td></tr></table></figure></li><li><p>清除机器配置状态，重新设置一个新账户（开机按住command+s；）</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">/sbin/mount -uaw</span><br><span class="line">rm var/db/.applesetupdone</span><br><span class="line">reboot</span><br></pre></td></tr></table></figure></li></ol><h2 id="Mac-开启任何来源选项"><a href="#Mac-开启任何来源选项" class="headerlink" title="Mac 开启任何来源选项"></a>Mac 开启任何来源选项</h2><figure class="highlight ada"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">sudo spctl <span class="comment">--master-disable</span></span><br></pre></td></tr></table></figure><h2 id="让系统更流畅"><a href="#让系统更流畅" class="headerlink" title="让系统更流畅"></a>让系统更流畅</h2><p>macOS 10.14 以后系统动画越来越复杂，对显卡要求比较高，导致配置比较老的设备运行起来很卡顿。<br>特别是big sur系统，使用intel核显的机器都不建议安装。</p><p>通过关闭一些动画和特效可以让系统更流畅些</p><p><img src="https://image.ponder.work/mweb/2021-10-10-16338756772783.jpg" alt></p><h2 id="键位修改"><a href="#键位修改" class="headerlink" title="键位修改"></a>键位修改</h2><p>使用 <a href="https://karabiner-elements.pqrs.org/" target="_blank" rel="noopener">karabiner</a> 可以对系统全局快捷键和应用快捷键进行修改。</p><p>通过配置 karabiner 将 command，control，option 按键进行重映射，可以让 macOS 的键位 和 Windows/Linux 接近。</p><h2 id="使-macOS-的命令风格与Linux相同"><a href="#使-macOS-的命令风格与Linux相同" class="headerlink" title="使 macOS 的命令风格与Linux相同"></a>使 macOS 的命令风格与Linux相同</h2><p>macOS 的命令应该是 Unix 风格的，和 Linux 有些不同，可选参数必须放在前面，会有些不方便。<br>比如<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> mac</span></span><br><span class="line">➜ ~ /bin/ls /tmp -al</span><br><span class="line">ls: -al: No such file or directory</span><br></pre></td></tr></table></figure></p><p>可以安装 gnu 工具，覆盖这些命令<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">brew install coreutils findutils gnu-tar htop</span><br><span class="line"><span class="built_in">export</span> PATH=<span class="string">"/usr/local/opt/coreutils/libexec/gnubin:<span class="variable">$PATH</span>"</span></span><br><span class="line"><span class="built_in">export</span> MANPATH=<span class="string">"/usr/local/opt/coreutils/libexec/gnuman:<span class="variable">$MANPATH</span>"</span></span><br></pre></td></tr></table></figure></p><h2 id="清除一些-macOS-系统默认快捷键"><a href="#清除一些-macOS-系统默认快捷键" class="headerlink" title="清除一些 macOS 系统默认快捷键"></a>清除一些 macOS 系统默认快捷键</h2><p>创建文件 ~/Library/KeyBindings/DefaultKeyBinding.dict<br>这里是清除了默认的 <code>Control + Command + 方向键</code>的行为</p><p>修改完成后必须重新打开现有app才能生效</p><figure class="highlight clojure"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"><span class="string">"^@\UF701"</span> = (<span class="string">"noop:"</span>)<span class="comment">;</span></span><br><span class="line"><span class="string">"^@\UF702"</span> = (<span class="string">"noop:"</span>)<span class="comment">;</span></span><br><span class="line"><span class="string">"^@\UF703"</span> = (<span class="string">"noop:"</span>)<span class="comment">;</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>字符含义</p><div class="table-container"><table><thead><tr><th>Prefix</th><th>Meaning</th></tr></thead><tbody><tr><td><code>~</code></td><td>⌥ Option key</td></tr><tr><td><code>$</code></td><td>⇧ Shift key</td></tr><tr><td><code>^</code></td><td>^ Control key</td></tr><tr><td><code>@</code></td><td>⌘ Command key</td></tr><tr><td><code>#</code></td><td>keys on number pad</td></tr></tbody></table></div><p>参考：</p><ul><li><a href="http://xahlee.info/kbd/osx_keybinding.html" target="_blank" rel="noopener">http://xahlee.info/kbd/osx_keybinding.html</a></li><li><a href="https://blog.victormendonca.com/2020/04/27/how-to-change-macos-key-bindings/" target="_blank" rel="noopener">https://blog.victormendonca.com/2020/04/27/how-to-change-macos-key-bindings/</a></li></ul><h2 id="关闭启动声音"><a href="#关闭启动声音" class="headerlink" title="关闭启动声音"></a>关闭启动声音</h2><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 关闭提示音</span></span><br><span class="line">sudo nvram <span class="attribute">StartupMute</span>=%01</span><br><span class="line"></span><br><span class="line"><span class="comment"># 打开duang</span></span><br><span class="line">sudo nvram <span class="attribute">StartupMute</span>=%00</span><br></pre></td></tr></table></figure><h2 id="crontab-文件路径"><a href="#crontab-文件路径" class="headerlink" title="crontab 文件路径"></a>crontab 文件路径</h2><p>用户执行<code>crontab -e</code>之后定时任务存储位置<code>/private/var/at/tabs/<username></code></p><h2 id="hostname-更改"><a href="#hostname-更改" class="headerlink" title="hostname 更改"></a>hostname 更改</h2><p>macos 默认没有设置hostname，导致终端<code>PS1</code>显示时会用网卡的mac地址作为主机名，看起来比较别扭。</p><p>可以通过<code>sudo scutil --set HostName <name></code>设置想要的主机名</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">➜ ~ hostname</span><br><span class="line">a8a159010000</span><br><span class="line">➜ ~ scutil --<span class="builtin-name">get</span> HostName</span><br><span class="line">HostName: <span class="keyword">not</span> set</span><br><span class="line">➜ ~ sudo scutil --<span class="builtin-name">set</span> HostName Mac-mini</span><br><span class="line">Password:</span><br><span class="line">➜ ~ hostname</span><br><span class="line">ruandeMac-mini</span><br></pre></td></tr></table></figure><h2 id="catalina-禁用系统更新提示"><a href="#catalina-禁用系统更新提示" class="headerlink" title="catalina 禁用系统更新提示"></a>catalina 禁用系统更新提示</h2><p>在终端中输入<br><figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="selector-tag">defaults</span> <span class="selector-tag">write</span> <span class="selector-tag">com</span><span class="selector-class">.apple</span><span class="selector-class">.systempreferences</span> <span class="selector-tag">AttentionPrefBundleIDs</span> 0 ; <span class="selector-tag">killall</span> <span class="selector-tag">Dock</span></span><br></pre></td></tr></table></figure></p><p>在 /etc/hosts 中加入<br><figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># <span class="selector-tag">disable</span> <span class="selector-tag">mac</span> <span class="selector-tag">update</span></span><br><span class="line">0<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.0</span> <span class="selector-tag">swdist</span><span class="selector-class">.apple</span><span class="selector-class">.com</span><span class="selector-class">.edgekey</span><span class="selector-class">.net</span></span><br><span class="line">0<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.0</span> <span class="selector-tag">swdist</span><span class="selector-class">.apple</span><span class="selector-class">.com</span><span class="selector-class">.akadns</span><span class="selector-class">.net</span></span><br></pre></td></tr></table></figure></p><h2 id="固定dock位置，防止在不同屏幕间移动"><a href="#固定dock位置，防止在不同屏幕间移动" class="headerlink" title="固定dock位置，防止在不同屏幕间移动"></a>固定dock位置，防止在不同屏幕间移动</h2><p>不是100%有效<br><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">defaults write com<span class="selector-class">.apple</span><span class="selector-class">.Dock</span> <span class="attribute">position</span>-immutable -bool yes; killall Dock</span><br></pre></td></tr></table></figure></p><h2 id="不生成-开头的隐藏文件"><a href="#不生成-开头的隐藏文件" class="headerlink" title="不生成 ._ 开头的隐藏文件"></a>不生成 <code>._</code> 开头的隐藏文件</h2><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">defaults write com<span class="selector-class">.apple</span><span class="selector-class">.desktopservices</span> DSDontWriteNetworkStores -bool true</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>对程序员来说，macOS 就是一个桌面支持比较好的 Linux/Unix，给日常开发带来了许多便利</p>
<p>本文记录一些日常使用中的小技巧。<br>
</summary>
<category term="工作生活" scheme="http://ponder.work/categories/%E5%B7%A5%E4%BD%9C%E7%94%9F%E6%B4%BB/"/>
<category term="mac" scheme="http://ponder.work/tags/mac/"/>
</entry>
<entry>
<title>Windows 网络共享</title>
<link href="http://ponder.work/2021/10/10/windows-network-share/"/>
<id>http://ponder.work/2021/10/10/windows-network-share/</id>
<published>2021-10-10T07:00:00.000Z</published>
<updated>2024-07-09T13:50:24.832Z</updated>
<content type="html"><![CDATA[<p>要在不暴露 client 的情况下共享网络，一般就只能使用 nat(Network Address Translation), linux 下可以使用 iptables 很轻松地搞定。<br>nat 包含 DNAT 和 SNAT, 要想双向互通，必须两者都实现。</p><p>windows下的网络共享只有SNAT那一部分，比如各自免费wifi软件。少了DNAT，外部网络就无法访问内部。</p><p>还好windows下可以配置端口转发，实现等效的DNAT<br><a id="more"></a></p><h2 id="先配置网络共享"><a href="#先配置网络共享" class="headerlink" title="先配置网络共享"></a>先配置网络共享</h2><p>前置要求，需要两张网卡，无线或者有线均可。</p><ul><li>A: 用于访问外网</li><li>B: 共享网络接入点</li></ul><p>打开：控制面板 》网络和Internet 》网络和共享中心 》更改适配器设置</p><p><img src="https://image.ponder.work/mweb/2021-10-10-16338760513347.jpg" alt></p><p><img src="https://image.ponder.work/mweb/2021-10-10-16338761455929.jpg" alt></p><p>右键A网卡 > 属性 > 共享 > 勾选允许 （win10可能有下拉选择，下拉选中B网卡）<br><img src="https://image.ponder.work/mweb/2021-10-10-16338761903207.jpg" alt></p><h2 id="端口映射配置"><a href="#端口映射配置" class="headerlink" title="端口映射配置"></a>端口映射配置</h2><p>netsh是Windows自带的端口转发/端口映射工具。</p><p>支持IPv4和IPv6,命令即时生效,重启系统后配置仍然存在。</p><h3 id="常用命令"><a href="#常用命令" class="headerlink" title="常用命令"></a>常用命令</h3><ul><li>add - 在一个表格中添加一个配置项。</li><li>delete - 从一个表格中删除一个配置项。</li><li>dump - 显示一个配置脚本。</li><li>help - 显示命令列表。</li><li>reset - 重置端口代理配置状态。</li><li>set - 设置配置信息。</li><li>show - 显示信息。</li></ul><h3 id="用法-以v4tov4为例"><a href="#用法-以v4tov4为例" class="headerlink" title="用法(以v4tov4为例)"></a>用法(以v4tov4为例)</h3><figure class="highlight inform7"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">add v4tov4 <span class="comment">[listenport=]</span>integer>|servicename> \</span><br><span class="line"> <span class="comment">[connectaddress=]</span>IPv4 address>|hostname> \</span><br><span class="line"> <span class="comment">[<span class="comment">[connectport=]</span>integer>|servicename>]</span> \</span><br><span class="line"> <span class="comment">[<span class="comment">[listenaddress=]</span>IPv4 address>|hostname>]</span> \</span><br><span class="line"> <span class="comment">[<span class="comment">[protocol=]</span>tcp]</span></span><br></pre></td></tr></table></figure><h3 id="参数说明"><a href="#参数说明" class="headerlink" title="参数说明"></a>参数说明</h3><ul><li>listenport - IPv4 侦听端口。</li><li>connectaddress - IPv4 连接地址。</li><li>connectport - IPv4 连接端口。</li><li>listenaddress - IPv4 侦听地址。</li><li>protocol - 使用的协议。现在只支持 TCP</li></ul><h3 id="案例-ssh端口转发"><a href="#案例-ssh端口转发" class="headerlink" title="案例(ssh端口转发)"></a>案例(ssh端口转发)</h3><p>将192.168.8.108的22端口映射到本地的2222端口</p><p>这样外部就可以通过本地的对外ip来ssh访问192.168.8.108了<br><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">netsh<span class="built_in"> interface </span>portproxy <span class="builtin-name">add</span> v4tov4 <span class="attribute">listenport</span>=2222 <span class="attribute">connectaddress</span>=192.168.8.108 <span class="attribute">connectport</span>=22</span><br></pre></td></tr></table></figure></p><h3 id="显示端口转发"><a href="#显示端口转发" class="headerlink" title="显示端口转发"></a>显示端口转发</h3><p>一般情况下使用下列命令进行查看</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">netsh<span class="built_in"> interface </span>portproxy show all</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<p>要在不暴露 client 的情况下共享网络，一般就只能使用 nat(Network Address Translation), linux 下可以使用 iptables 很轻松地搞定。<br>nat 包含 DNAT 和 SNAT, 要想双向互通，必须两者都实现。</p>
<p>windows下的网络共享只有SNAT那一部分，比如各自免费wifi软件。少了DNAT，外部网络就无法访问内部。</p>
<p>还好windows下可以配置端口转发，实现等效的DNAT<br>
</summary>
<category term="工作生活" scheme="http://ponder.work/categories/%E5%B7%A5%E4%BD%9C%E7%94%9F%E6%B4%BB/"/>
<category term="Windows" scheme="http://ponder.work/tags/Windows/"/>
</entry>
<entry>
<title>Docker mtu 引发的加班血案</title>
<link href="http://ponder.work/2021/09/21/docker-mtu-issues/"/>
<id>http://ponder.work/2021/09/21/docker-mtu-issues/</id>
<published>2021-09-21T10:00:00.000Z</published>
<updated>2024-07-09T13:50:24.830Z</updated>
<content type="html"><![CDATA[<p>最近在搞 torch 的工程化，基于 brpc 和 libtorch，将两者编译在一起的过程也是坑深，容下次再表。</p><p>为了简化部署，brpc 服务在 Docker 容器中运行。本地测试时功能一切正常，上到预发布环境时请求全部超时。</p><p>由于业务代码，brpc，docker环境，机房都是新的，在排查问题的过程中简直一头雾水。（当然根本原因还是水平不足）<br><a id="more"></a></p><h2 id="使尽浑身解数定位"><a href="#使尽浑身解数定位" class="headerlink" title="使尽浑身解数定位"></a>使尽浑身解数定位</h2><p>发现请求超时后，开始用CURL测试接口，用真实数据验证发现请求都耗时1s，这和用c++的预期完全不符。</p><h3 id="我代码不应该有bug"><a href="#我代码不应该有bug" class="headerlink" title="我代码不应该有bug!!"></a>我代码不应该有bug!!</h3><p>首先是怀疑业务代码有问题，逐行统计业务代码耗时，发现业务代码仅耗时10+ms。</p><h3 id="curl-不应该-Expect"><a href="#curl-不应该-Expect" class="headerlink" title="curl 不应该 Expect"></a>curl 不应该 Expect</h3><p>用空数据访问接口，发现耗时也只有20+ms，这时开始怀疑brpc是不是编译得有问题，或者说和libtorch编译到一起不兼容。<br>这时我请教了一位同事，他对brpc比较熟悉，然后他说是curl实现的问题，和brpc没关系, 参考<a href="https://github.com/apache/incubator-brpc/issues/1075" target="_blank" rel="noopener"> brpc issue</a></p><figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">curl传输的时候，会设置 <span class="string">Expect:</span> <span class="number">100</span>-<span class="keyword">continue</span>, 这个协议brpc本身没有实现, 所以curl会等待一个超时。</span><br><span class="line">加上 -H <span class="string">'Expect:'</span> 可以解决这个问题</span><br></pre></td></tr></table></figure><p>所以这个1s超时是个烟雾弹，线上client是Python，不会有这个问题。</p><h3 id="难道是-lvs"><a href="#难道是-lvs" class="headerlink" title="难道是 lvs ?"></a>难道是 lvs ?</h3><p>接着又是一通疯狂测试，各种角度体位测试。发现本机测试是ok的，透过lvs请求（跨机房）就会卡住直到超时，而且小body请求一切正常，大body请求卡住。</p><p>这时又开始怀疑brpc编译的不对，导致这个超时（brpc编译过程比较曲折，导致我不太有信心）。</p><p>于是我在这个服务器Docker中运行了另一个brpc服务，发现是一样的问题。</p><p>为了确认是否是brpc的问题，又写了个Python 的 echo server 进行测试，发现在docker中是一样的问题，但是不在docker中运行有一切正常。</p><p>这是时就可以确定，与代码无关，是docker或者lvs的问题，一度陷入僵局。</p><h3 id="docker-才是罪魁祸首"><a href="#docker-才是罪魁祸首" class="headerlink" title="docker 才是罪魁祸首"></a>docker 才是罪魁祸首</h3><p>完全没思路，于是找来了运维的同学，这时运维提了一下，这个机房的mtu会小一点，于是一切串起来了。</p><p>马上测试，发现机房的mtu只有1450，而docker0网桥的默认mtu是1500，这就很能解释小body没问题，大body卡死。</p><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">~# netstat -i</span><br><span class="line">Kernel Interface table</span><br><span class="line">Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg</span><br><span class="line">docker0 <span class="number">1500</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> BMU</span><br><span class="line">eth0 <span class="number">1450</span> <span class="number">1023246</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">110268</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> BMRU</span><br><span class="line">lo <span class="number">65536</span> <span class="number">1967</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">1967</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> LRU</span><br><span class="line">wlan0 <span class="number">1500</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span> BMU</span><br></pre></td></tr></table></figure><p>修改docker0的mtu，重启docker.service，一切问题都解决了。</p><figure class="highlight dts"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">cat <span class="params"><< EOF ></span> <span class="meta-keyword">/etc/</span>docker/daemon.json</span><br><span class="line">{</span><br><span class="line"> <span class="string">"mtu"</span>: <span class="number">1450</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">EOF</span><br></pre></td></tr></table></figure><h2 id="事后诸葛"><a href="#事后诸葛" class="headerlink" title="事后诸葛"></a>事后诸葛</h2><h3 id="找到问题等于解决了一半"><a href="#找到问题等于解决了一半" class="headerlink" title="找到问题等于解决了一半"></a>找到问题等于解决了一半</h3><p>这句话说得并不太懂，应该是“找到问题等于解决了90%”。<br>在互联网时代，找到了具体问题，在一通Google，基本等于解决了问题。你始终要确信，这个问题不应该只有你遇到。</p><p>在这个例子上，curl的误导大概花了一半的时间去定位。所以，定位问题首先得明确问题，如医生看病一样，确认问题发生的现象（卡住），位置(docker容器中)、程度（永久）、触发原因（大请求body）。</p><h3 id="要善于运用他人的经验和知识"><a href="#要善于运用他人的经验和知识" class="headerlink" title="要善于运用他人的经验和知识"></a>要善于运用他人的经验和知识</h3><p>找专业人士寻求帮助是非常高效的，会大大缩短定位问题的时间，因为他们会运用经验和知识快速排除错误选项。</p><h3 id="知识可能会误导你"><a href="#知识可能会误导你" class="headerlink" title="知识可能会误导你"></a>知识可能会误导你</h3><p>你所拥有的知识，并不是究竟的知识，也就是它所能应用的范围并不适应当前的场景，还有可能误导你。<br>你所缺失的知识，让你看不清前方正确的道路。</p><p>按我的知识，docker0是网桥，等价于交换机，除了性能问题，不应该导致丢包，就一直没往这个方向考虑（当然这块的知识也不扎实）。<br>MTU也不应该导致丢包，交换机应该会进行IP分片。<br>但忽略了一个点，虚拟的网桥并不是硬件网桥，可能并没有实现IP分片的逻辑（仅丢弃），又或者没有实现 PMTU（Path MTU Discovery）。<br>上面这点存疑，但更直接的原因是服务的IP包的 <code>Don't fragment</code> flag 为1，也就是禁止分片（为什么设置还不清楚）。<br><img src="https://image.ponder.work/mweb/2022-07-11-16575342644060.jpg" alt></p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://github.com/apache/incubator-brpc/issues/1075" target="_blank" rel="noopener">https://github.com/apache/incubator-brpc/issues/1075</a></li><li><a href="https://mp.weixin.qq.com/s/-Q1AkxUr9xzGKwUMV-FQhQ" target="_blank" rel="noopener">云网络丢包故障定位全景指南</a></li><li><a href="https://mlohr.com/docker-mtu/" target="_blank" rel="noopener">https://mlohr.com/docker-mtu/</a></li><li><a href="https://github.com/docker/for-win/issues/1144" target="_blank" rel="noopener">https://github.com/docker/for-win/issues/1144</a></li><li><a href="https://www.cnblogs.com/sammyliu/p/5079898.html" target="_blank" rel="noopener">https://www.cnblogs.com/sammyliu/p/5079898.html</a></li><li><a href="https://github.com/moby/moby/issues/12565" target="_blank" rel="noopener">https://github.com/moby/moby/issues/12565</a></li></ul>]]></content>
<summary type="html">
<p>最近在搞 torch 的工程化，基于 brpc 和 libtorch，将两者编译在一起的过程也是坑深，容下次再表。</p>
<p>为了简化部署，brpc 服务在 Docker 容器中运行。本地测试时功能一切正常，上到预发布环境时请求全部超时。</p>
<p>由于业务代码，brpc，docker环境，机房都是新的，在排查问题的过程中简直一头雾水。（当然根本原因还是水平不足）<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Docker" scheme="http://ponder.work/tags/Docker/"/>
</entry>
<entry>
<title>Hyper-V 中安装 openmediavault(OMV) 实现完美 NAS 文件共享</title>
<link href="http://ponder.work/2021/09/21/openmediavault-on-hyperv/"/>
<id>http://ponder.work/2021/09/21/openmediavault-on-hyperv/</id>
<published>2021-09-21T09:00:00.000Z</published>
<updated>2024-07-09T13:50:24.797Z</updated>
<content type="html"><![CDATA[<p>一直在寻找适合自己的 NAS 存储方案，也做过一些尝试，可总有些不完美的点。</p><p>最近终于找到一个接近完美的方案：<code>Hyper-V + NFS + openmediavault</code></p><a id="more"></a><h2 id="思路"><a href="#思路" class="headerlink" title="思路"></a>思路</h2><p>我对 NAS 几点硬性要求</p><ol><li>能支持 NTFS 文件系统，否则现有数据迁移成本很高</li><li>文件共享必须支持回收站，否则数据易丢失</li><li>文件检索要方便</li></ol><p>所以之前一直用 windows 作为 NAS 系统，最大的问题是不支持共享文件夹的回收站。</p><p>现有 NAS 方案的对比</p><div class="table-container"><table><thead><tr><th></th><th>good</th><th>bad</th></tr></thead><tbody><tr><td>windows smb</td><td>性能</td><td>无法扩展功能（如smb回收站，timemachine）</td></tr><tr><td>wsl1 samba</td><td>性能略低于smb，功能可扩展</td><td>部分共享文件无法打开, 配置略繁琐</td></tr><tr><td>wsl2 samba（omv）</td><td>功能可扩展</td><td>无法解决桥接网络，需要更好的cpu</td></tr><tr><td>nfs</td><td>性能最强</td><td>无法扩展功能</td></tr><tr><td>群晖 / OMV / FreeNAS</td><td>功能多，功能可扩展</td><td>无法管理底层文件</td></tr><tr><td>windows + nfs + vmware(群晖)</td><td>可管理底层文件，功能可扩展</td><td>物理机休眠虚拟机无法唤醒</td></tr><tr><td>windows + nfs + hyper-v(omv）</td><td>功能可扩展，性能接近smb，可休眠</td><td>需要更好的cpu，hyperv使用略繁琐</td></tr></tbody></table></div><p>本方案的思路受之前文章的启发 <a href="/2021/01/02/install-openmediavault-in-WSL2/">在WSL2中安装openmediavault(OMV)</a></p><p>WSL2 的便利之处在于让 linux 系统能以一个足够快的速度访问 windows 文件，据了解是使用 9p(Plan 9 9p remote filesystem protocol) 协议进行文件共享。</p><p>那么我们只要在 Hyper-V 上运行虚拟机，并通过某种方式（这里选择NFS）共享文件到虚拟机里，就能绕开 WSL2 的缺点。<br>实现 NAS 的底层文件系统由 windows 管理， 上层文件共享等应用由虚拟机中的 NAS 系统处理，兼顾了稳定性和扩展性。</p><h2 id="方案"><a href="#方案" class="headerlink" title="方案"></a>方案</h2><p>软件环境选择</p><ul><li>windows 10 21H1</li><li>haneWIN NFS server 1.2.59</li><li>openmediavault</li></ul><h3 id="Windows-10-安装及配置"><a href="#Windows-10-安装及配置" class="headerlink" title="Windows 10 安装及配置"></a>Windows 10 安装及配置</h3><ol><li>设置开机自动登录用户（可选）: <a href="https://zhuanlan.zhihu.com/p/61262940" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/61262940</a></li><li>禁止自动从睡眠中唤醒（可选）: 控制面板 -> 电源选项 -> 更改计划设置 -> 更改高级计划设置 -> 睡眠 -> 允许使用睡眠唤醒定时器（禁用）</li><li>启用 Hyper-V：<code>DISM /Online /Enable-Feature /All /FeatureName:Microsoft-Hyper-V</code></li><li>Hyper-V 配置桥接网络：虚拟交换机管理器 -> 新建虚拟网络交换机(外部)</li><li>安装 haneWIN NFS server 并配置需要共享的硬盘</li></ol><h3 id="openmediavault-安装及配置"><a href="#openmediavault-安装及配置" class="headerlink" title="openmediavault 安装及配置"></a>openmediavault 安装及配置</h3><ol><li>创建 Hyper-V 虚拟机：选择第一代，选择桥接网卡，虚拟硬盘需挂载到 IDE 控制器</li><li>挂载安装镜像，安装 openmediavault 系统</li><li><p>安装 omv-extras</p> <figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget -O - https:<span class="regexp">//gi</span>thub.com<span class="regexp">/OpenMediaVault-Plugin-Developers/</span>packages<span class="regexp">/raw/m</span>aster<span class="regexp">/install | bash</span></span><br></pre></td></tr></table></figure></li><li><p>安装插件：openmediavault-sharerootfs(共享系统分区的文件夹) openmediavault-remotemount(远程挂载)</p></li><li>挂载 NFS 共享：存储器 -> 远程挂载</li><li>建立共享文件夹: 选择系统分区上的 NFS 挂载文件夹</li><li>启用smb共享</li><li>关闭虚拟机，并配置虚拟机的网卡为静态MAC地址（如果是导入的虚拟机，注意检查MAC地址是否合法）</li></ol><p><img src="https://image.ponder.work/mweb/2021-09-21-16322275739423.jpg" alt="-w610"><br><img src="https://image.ponder.work/mweb/2021-09-21-16322277511161.jpg" alt="-w524"></p><h3 id="其他优化配置"><a href="#其他优化配置" class="headerlink" title="其他优化配置"></a>其他优化配置</h3><h4 id="samba-性能优化"><a href="#samba-性能优化" class="headerlink" title="samba 性能优化"></a>samba 性能优化</h4><p>配置页面：服务 -> SMB/CIFS -> 设置 -> 高级设置 -> 扩展选项</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">veto files = /.Trashes/<span class="variable">$RECYCLE</span>.BIN<span class="built_in">/System </span>Volume Information/.recycle/</span><br><span class="line">socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE <span class="attribute">SO_RCVBUF</span>=98304 <span class="attribute">SO_SNDBUF</span>=98304</span><br><span class="line">dead time = 30</span><br><span class="line">getwd cache = <span class="literal">yes</span></span><br><span class="line">min protocol = SMB2</span><br><span class="line"><span class="comment">#max protocol = SMB2</span></span><br><span class="line">aio read size = 40960</span><br><span class="line">aio write size = 40960</span><br><span class="line">write cache size = 262144</span><br><span class="line">large readwrite = <span class="literal">yes</span></span><br><span class="line">fake oplocks = <span class="literal">yes</span></span><br><span class="line">oplocks = <span class="literal">no</span></span><br></pre></td></tr></table></figure><h4 id="samba-回收站优化"><a href="#samba-回收站优化" class="headerlink" title="samba 回收站优化"></a>samba 回收站优化</h4><p>默认的回收站不能按天对文件进行分类，不方便进行清理</p><p>配置页面：服务 -> SMB/CIFS -> 共享 -> 共享文件夹<br>启用回收站选项，并在扩展选项中增加</p><figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">recycle:</span>repository = .recycle<span class="regexp">/%U/</span>today</span><br></pre></td></tr></table></figure><p>配置页面：系统 -> 计划任务<br>增加共享文件夹回收站整理脚本<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line"></span><br><span class="line"><span class="built_in">set</span> -x -e</span><br><span class="line"></span><br><span class="line">BASE=/srv/nfs_root</span><br><span class="line">TODAY=$(date +<span class="string">'%Y%m%d'</span>)</span><br><span class="line">USERS=(</span><br><span class="line"> data</span><br><span class="line">)</span><br><span class="line"><span class="comment"># read -r -a USERS <<< $(groupmems -g users -l)</span></span><br><span class="line"></span><br><span class="line"><span class="built_in">cd</span> <span class="variable">$BASE</span></span><br><span class="line"><span class="keyword">for</span> share <span class="keyword">in</span> $(ls); <span class="keyword">do</span></span><br><span class="line"> <span class="keyword">for</span> user <span class="keyword">in</span> <span class="variable">${USERS[@]}</span>; <span class="keyword">do</span></span><br><span class="line"> rec=<span class="string">"<span class="variable">$BASE</span>/<span class="variable">${share}</span>/.recycle/<span class="variable">$user</span>"</span></span><br><span class="line"> <span class="keyword">if</span> [[ ! -d <span class="variable">$rec</span> ]]; <span class="keyword">then</span></span><br><span class="line"> mkdir -p <span class="variable">$rec</span></span><br><span class="line"> <span class="keyword">fi</span></span><br><span class="line"> <span class="built_in">cd</span> <span class="variable">$rec</span></span><br><span class="line"> <span class="keyword">if</span> [[ ! -d <span class="variable">$TODAY</span> ]]; <span class="keyword">then</span></span><br><span class="line"> mkdir <span class="variable">$TODAY</span></span><br><span class="line"> <span class="keyword">fi</span></span><br><span class="line"> <span class="keyword">if</span> [[ -L today || -f today ]]; <span class="keyword">then</span></span><br><span class="line"> rm -f today</span><br><span class="line"> <span class="keyword">elif</span> [[ -d today ]]; <span class="keyword">then</span></span><br><span class="line"> mv today <span class="variable">$TODAY</span>/$(date +<span class="string">'bak.%s'</span>)</span><br><span class="line"> <span class="keyword">fi</span></span><br><span class="line"> ln -s ./<span class="variable">$TODAY</span> today</span><br><span class="line"> <span class="keyword">done</span></span><br><span class="line"><span class="keyword">done</span></span><br></pre></td></tr></table></figure></p><p>windows 系统中，增加定时任务，删除NFS共享目录中的回收站文件到系统回收站<br>Python 代码如下，超过一定时间的文件会被送到 windows 回收站</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> time</span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> logging</span><br><span class="line"><span class="keyword">from</span> datetime <span class="keyword">import</span> datetime, timedelta</span><br><span class="line"><span class="keyword">from</span> pathlib <span class="keyword">import</span> Path</span><br><span class="line"><span class="keyword">from</span> send2trash <span class="keyword">import</span> send2trash</span><br><span class="line"></span><br><span class="line">os.chdir(os.path.dirname(__file__))</span><br><span class="line"></span><br><span class="line">logging.basicConfig(</span><br><span class="line"> format=<span class="string">'[%(levelname)s %(asctime)s %(filename)s:%(lineno)d]\t%(message)s'</span>,</span><br><span class="line"> level=<span class="string">'INFO'</span>,</span><br><span class="line"> filename=<span class="string">'log/recycle.log'</span>,</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line">EXPORTS_FILE=<span class="string">'D:\\nas_file\\nfsd\\exports'</span></span><br><span class="line">KEEP_DAYS = <span class="number">30</span></span><br><span class="line">USERS = [<span class="string">'data'</span>]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">get_size</span><span class="params">(path: str)</span> -> int:</span></span><br><span class="line"> <span class="keyword">return</span> sum(p.stat().st_size <span class="keyword">for</span> p <span class="keyword">in</span> Path(path).rglob(<span class="string">'*'</span>))</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line"> exports = []</span><br><span class="line"> <span class="keyword">for</span> line <span class="keyword">in</span> open(EXPORTS_FILE):</span><br><span class="line"> parts = line.strip().split()</span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> parts <span class="keyword">or</span> parts[<span class="number">0</span>].startswith(<span class="string">'#'</span>):</span><br><span class="line"> <span class="keyword">continue</span></span><br><span class="line"> exports.append(parts[<span class="number">0</span>])</span><br><span class="line"></span><br><span class="line"> dt = (datetime.now() - timedelta(days=KEEP_DAYS)).strftime(<span class="string">'%Y%m%d'</span>)</span><br><span class="line"> <span class="keyword">for</span> export <span class="keyword">in</span> exports:</span><br><span class="line"> <span class="keyword">for</span> user <span class="keyword">in</span> USERS:</span><br><span class="line"> rec = os.path.join(export, <span class="string">'.recycle'</span>, user)</span><br><span class="line"> <span class="keyword">if</span> os.path.exists(rec):</span><br><span class="line"> <span class="keyword">for</span> item <span class="keyword">in</span> os.listdir(rec):</span><br><span class="line"> path = os.path.join(rec, item)</span><br><span class="line"> <span class="comment"># print(path, get_size(path))</span></span><br><span class="line"> <span class="keyword">if</span> os.path.isdir(path) <span class="keyword">and</span> item.startswith(<span class="string">'2'</span>) <span class="keyword">and</span> item < dt:</span><br><span class="line"> size = get_size(path)</span><br><span class="line"> <span class="keyword">if</span> size <= <span class="number">0</span>:</span><br><span class="line"> logging.info(<span class="string">'rm empyt\t%s'</span>, path)</span><br><span class="line"> os.rmdir(path)</span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> logging.info(<span class="string">'trash\t%s'</span>, path)</span><br><span class="line"> send2trash(path)</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"> <span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line"> <span class="keyword">try</span>:</span><br><span class="line"> logging.info(<span class="string">'check'</span>)</span><br><span class="line"> main()</span><br><span class="line"> time.sleep(<span class="number">3600</span>)</span><br><span class="line"> <span class="keyword">except</span> Exception <span class="keyword">as</span> e:</span><br><span class="line"> logging.exception(e)</span><br></pre></td></tr></table></figure><h2 id="其他"><a href="#其他" class="headerlink" title="其他"></a>其他</h2><p>也可考虑将上文 openmediavalut 替换成黑群晖系统，牺牲定制化能力的同时大大增加易用性。<br>当然，你也可以两者双修</p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v" target="_blank" rel="noopener">https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v</a></li><li><a href="https://nelsonslog.wordpress.com/2019/06/01/wsl-access-to-linux-files-via-plan-9/" target="_blank" rel="noopener">https://nelsonslog.wordpress.com/2019/06/01/wsl-access-to-linux-files-via-plan-9/</a></li></ul>]]></content>
<summary type="html">
<p>一直在寻找适合自己的 NAS 存储方案，也做过一些尝试，可总有些不完美的点。</p>
<p>最近终于找到一个接近完美的方案：<code>Hyper-V + NFS + openmediavault</code></p>
</summary>
<category term="工作生活" scheme="http://ponder.work/categories/%E5%B7%A5%E4%BD%9C%E7%94%9F%E6%B4%BB/"/>
<category term="NAS" scheme="http://ponder.work/tags/NAS/"/>
<category term="Hyper-V" scheme="http://ponder.work/tags/Hyper-V/"/>
</entry>
<entry>
<title>使用 iTerm2 管理 Tmux 会话</title>
<link href="http://ponder.work/2021/08/02/iterm2-tmux-integration/"/>
<id>http://ponder.work/2021/08/02/iterm2-tmux-integration/</id>
<published>2021-08-02T12:00:00.000Z</published>
<updated>2024-07-09T13:50:24.797Z</updated>
<content type="html"><![CDATA[<p>Tmux 是一个终端复用器（terminal multiplexer），非常有用，属于常用的开发工具。</p><p>一个典型的例子就是，SSH 登录远程计算机，打开一个远程窗口执行命令。这时，网络突然断线，再次登录的时候，是找不回上一次执行的命令的。因为上一次 SSH 会话已经终止了，里面的进程也随之消失了。</p><p>Tmux 可以维持和管理我们的远程终端会话，和服务断线重连后也不会丢失工作状态, 同时可以在一个终端连接中开启多个窗口（window）和窗格（pane）。<br><a id="more"></a><br>比如，下面就包含了2个窗口和3个窗格<br><img src="https://image.ponder.work/mweb/2021-08-03-16279763351692.jpg" alt></p><p>具体 Tmux 的细节和使用可以参考 <a href="https://www.ruanyifeng.com/blog/2019/10/tmux.html" target="_blank" rel="noopener">阮一峰的文章</a></p><p>但是 Tmux 也存在以下几个问题（个人观点）</p><ol><li>窗口和会话管理默认是全键盘操作，需要记比较多快捷键</li><li>由于窗口是 Tmux 虚拟的, 不支持文本回滚（scrollback），文本复制不完美</li><li>在 mouse mode 下，无法复制文本，非 mouse mode，调整窗格比较麻烦</li><li>不支持rzsz</li></ol><p>而 iTerm2 内置了 Tmux 绑定功能，可以将 tmux 的窗口和窗格映射成原生的窗口和窗格，可以用 iTerm2 的菜单和快捷键来操作窗口，解决了前3点问题。</p><p>至于第4点，rzsz 由于 tmux 的实现机制决定了是无解的。<br>然而 <a href="https://github.com/lonnywong" target="_blank" rel="noopener">lonnywong</a> 实现了替代方案 <a href="https://github.com/trzsz/trzsz" target="_blank" rel="noopener">trzsz</a> ，<strong>完美解决了文件上传下载的问题</strong>，亲测非常好用。</p><h2 id="iTerm2-配置"><a href="#iTerm2-配置" class="headerlink" title="iTerm2 配置"></a>iTerm2 配置</h2><p>可以对 tmux 窗口的映射进行一些定制</p><p>iTerm2 对于 tmux 会话有一个profile，建议对终端颜色和外观进行一些定制化，将原生窗口和 tmux 窗口区分开来。</p><p><img src="https://image.ponder.work/mweb/2021-08-03-16279767130639.jpg" alt><br><img src="https://image.ponder.work/mweb/2021-08-03-16279767560503.jpg" alt></p><h2 id="使用"><a href="#使用" class="headerlink" title="使用"></a>使用</h2><p>对于终端机器的 Tmux 版本有要求，需要支持<code>-CC</code>命令</p><p><strong>具体方法</strong></p><ul><li>新建tmux会话: <code>tmux -CC</code></li><li>断开重连(attach): <code>tmux -CC attach</code></li><li>如已存在会话则连接，否则新建会话：<code>tmux -CC new -A -s main</code></li><li>如果是ssh连接的机器: <code>ssh -t user@host 'tmux -CC new -A -s main'</code></li><li>断开连接(dettach): 在连接的窗口按 esc，或者直接关掉连接的 tab 吧</li><li>关掉 session(destroy): 关闭当前 session 的所有 tab 即可</li></ul><p>还可以使用 iTerm 的 tmux dashboard 来管理多个会话。</p><p><img src="https://image.ponder.work/mweb/2021-08-03-16279791270158.jpg" alt></p><p><img src="https://image.ponder.work/mweb/2021-08-03-16279791503013.jpg" alt></p><p><img src="https://image.ponder.work/mweb/2021-08-03-16279775548246.jpg" alt></p><h2 id="其他"><a href="#其他" class="headerlink" title="其他"></a>其他</h2><ul><li>tmux静态编译版本 <a href="https://github.com/mjakob-gh/build-static-tmux/releases" target="_blank" rel="noopener">https://github.com/mjakob-gh/build-static-tmux/releases</a></li><li>tmux显示颜色 <code>echo 'set -g default-terminal "screen-256color"' >> ~/.tmux.conf</code></li></ul><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://www.ruanyifeng.com/blog/2019/10/tmux.html" target="_blank" rel="noopener">https://www.ruanyifeng.com/blog/2019/10/tmux.html</a></li><li><a href="https://www.v2ex.com/t/589453" target="_blank" rel="noopener">https://www.v2ex.com/t/589453</a></li><li><a href="https://iterm2.com/documentation-tmux-integration.html" target="_blank" rel="noopener">https://iterm2.com/documentation-tmux-integration.html</a></li><li><a href="https://gitlab.com/gnachman/iterm2/-/wikis/tmux-Integration-Best-Practices" target="_blank" rel="noopener">https://gitlab.com/gnachman/iterm2/-/wikis/tmux-Integration-Best-Practices</a></li></ul>]]></content>
<summary type="html">
<p>Tmux 是一个终端复用器（terminal multiplexer），非常有用，属于常用的开发工具。</p>
<p>一个典型的例子就是，SSH 登录远程计算机，打开一个远程窗口执行命令。这时，网络突然断线，再次登录的时候，是找不回上一次执行的命令的。因为上一次 SSH 会话已经终止了，里面的进程也随之消失了。</p>
<p>Tmux 可以维持和管理我们的远程终端会话，和服务断线重连后也不会丢失工作状态, 同时可以在一个终端连接中开启多个窗口（window）和窗格（pane）。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="tmux" scheme="http://ponder.work/tags/tmux/"/>
</entry>
<entry>
<title>Redis主从复制机制</title>
<link href="http://ponder.work/2021/07/21/redis-master-slave/"/>
<id>http://ponder.work/2021/07/21/redis-master-slave/</id>
<published>2021-07-21T14:48:00.000Z</published>
<updated>2024-07-09T13:50:24.829Z</updated>
<content type="html"><![CDATA[<p>Redis作为内存型数据，为了高可用，必须有数据备份，这里采取主从的模式。<br>用户可以通过执行 SLAVEOF 命令或者设置 slaveof 选项，让一个服务器去复制 （replicate) 另一个服务器。<br><a id="more"></a><br>如果服务器 <code>127.0.0.1:12345</code> 向 <code>127.0.0.1:6379</code> 发送 <code>SLAVEOF 127.0.0.1 6379</code>, 则服务器（12345）将成为服务器（6379）的从服务器。</p><h2 id="旧版复制功能（全量复制）"><a href="#旧版复制功能（全量复制）" class="headerlink" title="旧版复制功能（全量复制）"></a>旧版复制功能（全量复制）</h2><p>Redis的复制功能分为同步（RDB文件）和命令传播（同步写命令）两个阶段，具体步骤如下。</p><ol><li>从服务器发送<code>SYNC</code></li><li>主服务器接收<code>SYNC</code>后执行<code>BGSAVE</code>生成RDB文件，同时用缓冲区记录之后所有写命令。</li><li>主服务器发送RDB到从服务器，从服务器加载RDB</li><li>主服务器发送缓冲区内的写命令</li><li>之后主服务器写命令都需要同时发一份给从服务器（命令传播）</li></ol><p><img src="https://image.ponder.work/mweb/2021-07-25-16272051223635.jpg" alt="-w908"></p><h2 id="缺陷及解决方案"><a href="#缺陷及解决方案" class="headerlink" title="缺陷及解决方案"></a>缺陷及解决方案</h2><p>由于RDB的生成发送非常耗时，主从短暂断线的情况下，也需要重复生成，主从同步的效率就非常低了。</p><p>Redis从2.8版本开始, 使用<code>PSYNC</code>命令替换<code>SYNC</code>，增加了部分同步功能，对断线重连的情况进行了优化。</p><p>主从服务器都记录了<strong>复制偏移量</strong>，记录了主服务器发出的字节数和从服务器收到的字节数,并且主服务器使用一个<strong>复制积压缓冲区</strong>记录最近发出的数据（FIFO）。</p><p>同步时，从服务器会发送复制偏移量。</p><ul><li>如果，主从偏移量相差的这部分数据在缓冲区中，则主服务器就发送这部分数据</li><li>否则，执行全量复制。</li></ul><p>同时，主服务器每次启动时会生成<strong>运行id</strong>，防止主服务器重启后复制混乱。</p><p><img src="https://image.ponder.work/mweb/2021-07-25-16272066103593.jpg" alt="-w586"><br><img src="https://image.ponder.work/mweb/2021-07-25-16272066397939.jpg" alt="-w552"></p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li>redis 3.0 源码</li><li>redis 设计与实现</li></ul>]]></content>
<summary type="html">
<p>Redis作为内存型数据，为了高可用，必须有数据备份，这里采取主从的模式。<br>用户可以通过执行 SLAVEOF 命令或者设置 slaveof 选项，让一个服务器去复制 （replicate) 另一个服务器。<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Redis" scheme="http://ponder.work/tags/Redis/"/>
</entry>
<entry>
<title>Redis持久化：RDB与AOF</title>
<link href="http://ponder.work/2021/07/20/redis-source-code-rdb-aof-persistence/"/>
<id>http://ponder.work/2021/07/20/redis-source-code-rdb-aof-persistence/</id>
<published>2021-07-20T14:48:00.000Z</published>
<updated>2024-07-09T13:50:24.832Z</updated>
<content type="html"><![CDATA[<p>Redis是内存型数据库，所有数据都存储在内存中。而内存是易失型存储，一旦进程退出所有数据都会丢失。</p><p>所谓持久化，就是将Redis在内存中的数据库状态以某次格式保存到磁盘里面，避免数据意外丢失。</p><p>Redis有两种持久化方式：RDB (Redis Database)、AOF (Append Only File)<br><a id="more"></a></p><h2 id="RDB持久化"><a href="#RDB持久化" class="headerlink" title="RDB持久化"></a>RDB持久化</h2><p>RDB持久化功能所生成的RDB文件是一个经过压缩的二进制文件，包含了Redis数据库的所有数据。</p><p>RDB是将redis中所有db中的所有键值对以如下格式进行储存</p><p><img src="https://image.ponder.work/mweb/2021-07-21-16268563558568.jpg" alt></p><h3 id="RDB文件创建"><a href="#RDB文件创建" class="headerlink" title="RDB文件创建"></a>RDB文件创建</h3><p>有两个命令可以生成RDB文件，<code>SAVE</code> 和 <code>BGSAVE</code>。生成RDB文件时，redis会遍历所有非空db的所有键值对按一定格式存储到RDB文件中。</p><p><code>SAVE</code> 命令会在当前进程进行，期间服务器会阻塞，不能处理任何请求。<br><code>BGSAVE</code> 命令会fork一个子进程来创建RDB（利用Copy-on-write），服务继续处理命令请求。</p><p>为了避免竞争条件和性能问题，<code>SAVE</code> 和 <code>BGSAVE</code>任意时刻只能有一个在执行。</p><p>用户可以通过save选项设置多个保存条件,但只要其中任意一个条件被满足，就会触发RDB保存.<br>save选项的格式是 <code>save seconds option_times</code>。例如<code>save 900 1</code>，若服务器在900秒之内, 对数据库进行了至少1次修改，则执行BGSAVE。</p><p>BGSAVE也已可能会阻塞请求，因为磁盘io满了，这时如果有fsync操作，服务也会阻塞。<br>可以设置 <code>no-appendfsync-on-rewrite yes</code>, 在子进程处理和写硬盘时, 主进程不调用 fsync() 操作。</p><h3 id="RDB文件读取"><a href="#RDB文件读取" class="headerlink" title="RDB文件读取"></a>RDB文件读取</h3><p>服务器启动时会自动载入RDB文件，Redis并没有专门用于载人RDB文件的命令。</p><p>如果服务器开启了AOF持久化功能，那么服务器会优先使用AOF文件来还原数据库状态。</p><h2 id="AOF持久化"><a href="#AOF持久化" class="headerlink" title="AOF持久化"></a>AOF持久化</h2><p>由于RDB生成的机制决定了，RDB文件总是会和redis内存有部分不一致，<strong>RDB文件会缺少从上次BGSAVE开始到当前时刻的所有改动</strong>。AOF持久化的存在就是为了解决该问题。</p><p>AOF持久化是通过保存执行的写命令来记录数据库状态。<br>因为Redis的命令请求协议是纯文本格式，所以AOF文件类似如下。<br><figure class="highlight tex"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">*2<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span><span class="formula">$6<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">nSELECT</span></span><span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span>$</span>l<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">nO</span></span><span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span></span><br><span class="line">*3<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span><span class="formula">$3<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">nSET</span></span><span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span>$</span>3<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">nmsg</span></span><span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span><span class="formula">$5<span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">nhello</span></span><span class="tag">\<span class="name">r</span></span><span class="tag">\<span class="name">n</span></span></span></span><br></pre></td></tr></table></figure></p><h3 id="AOF持久化实现"><a href="#AOF持久化实现" class="headerlink" title="AOF持久化实现"></a>AOF持久化实现</h3><ol><li>命令追加：执行完命令，将所有<strong>写命令</strong>追加到aof_buf缓冲区末尾</li><li>文件写入：将aof_buf的内容写入文件，并清空aof_buf</li><li>文件同步：清空完aof_buf之后，根据appendfsync配置的策略，决定如何刷新文件缓存到硬盘。</li></ol><p>appendfsync决定如何刷新文件缓存到硬盘，该选项的值直接影响的效率和安全性。<br>当故障停机时，文件缓冲区内的数据会丢失。<br>appendfsync有以下选项</p><ul><li>always: 每次都会进行文件缓冲区刷新，最安全，效率也最低。</li><li>no: 不主动刷新文件缓冲区，由系统决定刷新时机，安全性最低，效率最高</li><li>everysec: 每秒刷新一次文件缓冲区，安全性和效率的折中方案。</li></ul><h3 id="AOF文件还原"><a href="#AOF文件还原" class="headerlink" title="AOF文件还原"></a>AOF文件还原</h3><p>AOF的还原就是模仿客户端逐条执行文件里的命令。</p><p>AOF的还原时机也是服务启动时，并且在还原过程中能正常执行的只有 PUBSUB 等模块。</p><p>步骤如下：</p><ol><li>创建不带网络连接的伪客户端(fake client)</li><li>从AOF文件中分析和读取一条命令</li><li>使用伪客户端执行该命令</li><li>重复步骤2、3，直到处理完所有命令</li></ol><h3 id="AOF重写"><a href="#AOF重写" class="headerlink" title="AOF重写"></a>AOF重写</h3><p>由于AOF是直接记录的写命令而不是数据库状态，所以文件中包含很多冗余语句，导致文件膨胀。</p><p>比如下面的这些命令，其实最终数据库状态等价于<code>lpush numbers 333</code>, 前4条语句都是冗余的。<br><figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">127<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.1</span><span class="selector-pseudo">:6379</span>> <span class="selector-tag">lpush</span> <span class="selector-tag">numbers</span> 111</span><br><span class="line">(<span class="selector-tag">integer</span>) 1</span><br><span class="line">127<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.1</span><span class="selector-pseudo">:6379</span>> <span class="selector-tag">lpush</span> <span class="selector-tag">numbers</span> 222</span><br><span class="line">(<span class="selector-tag">integer</span>) 2</span><br><span class="line">127<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.1</span><span class="selector-pseudo">:6379</span>> <span class="selector-tag">lpop</span> <span class="selector-tag">numbers</span></span><br><span class="line">"222"</span><br><span class="line">127<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.1</span><span class="selector-pseudo">:6379</span>> <span class="selector-tag">lpop</span> <span class="selector-tag">numbers</span></span><br><span class="line">"111"</span><br><span class="line">127<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.1</span><span class="selector-pseudo">:6379</span>> <span class="selector-tag">lpush</span> <span class="selector-tag">numbers</span> 333</span><br><span class="line">(<span class="selector-tag">integer</span>) 1</span><br><span class="line">127<span class="selector-class">.0</span><span class="selector-class">.0</span><span class="selector-class">.1</span><span class="selector-pseudo">:6379</span>> <span class="selector-tag">lrange</span> <span class="selector-tag">numbers</span> 0 <span class="selector-tag">-1</span></span><br><span class="line">1) "333"</span><br></pre></td></tr></table></figure></p><p>为了解决AOF文件体积膨胀的问题，Redis提供了AOF文件重写(rewrite) 功能。</p><p>AOF文件重写是遍历redis的所有键值对，生成对应的redis命令，写入到一个新的文件中，并替换旧AOF文件。<br>所以AOF文件重写和旧AOF文件并没有关系，更应该称之为<strong>AOF重生成</strong>。</p><p>AOF重写程序在子进程里执行, 这样做可以同时达到两个目的:</p><ul><li>子进程进行AOF重写期间, 服务器进程可以继续处理命令请求。</li><li>子进程带有服务器进程的数据副本，可以在避免使用锁的情况下，保证数据的安全性（Copy-on-write）。</li></ul><p>在AOF重新过程中，所有命令会额外会写一份到<strong>AOF重写缓冲区</strong>中，当新AOF文件生成时，父进程会将AOF重写缓冲区的内容追加到新AOF文件中，并替换旧AOF文件。</p><p><img src="https://image.ponder.work/mweb/2021-07-21-16268637141893.jpg" alt></p><p>为防止AOF重写失败，AOF缓冲区在重写过程中依然正常工作。</p><p><img src="https://image.ponder.work/mweb/2021-07-21-16268635658876.jpg" alt></p><p><strong>注意</strong>：redis主从是基于<code>RDB + 命令传播</code>，并没有利用AOF文件，与MySQL的binlog不同。</p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li>redis 3.0 源码</li><li>redis 设计与实现</li></ul>]]></content>
<summary type="html">
<p>Redis是内存型数据库，所有数据都存储在内存中。而内存是易失型存储，一旦进程退出所有数据都会丢失。</p>
<p>所谓持久化，就是将Redis在内存中的数据库状态以某次格式保存到磁盘里面，避免数据意外丢失。</p>
<p>Redis有两种持久化方式：RDB (Redis Database)、AOF (Append Only File)<br>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Redis" scheme="http://ponder.work/tags/Redis/"/>
</entry>
<entry>
<title>数独与回溯法</title>
<link href="http://ponder.work/2021/07/10/sodoku-and-backtracking/"/>
<id>http://ponder.work/2021/07/10/sodoku-and-backtracking/</id>
<published>2021-07-10T11:48:00.000Z</published>
<updated>2024-07-09T13:50:24.832Z</updated>
<content type="html"><![CDATA[<p>数独是一种数学逻辑游戏，游戏由9×9个格子组成，玩家需要根据格子提供的数字推理出其他格子的数字。游戏设计者会提供最少17个数字使得解答谜题只有一个答案。</p><p>数独的解法需 遵循如下规则：</p><ol><li>数字 1-9 在每一行只能出现一次。</li><li>数字 1-9 在每一列只能出现一次。</li><li>数字 1-9 在每一个以粗实线分隔的 3x3 宫内只能出现一次。<a id="more"></a><img src="https://image.ponder.work/mweb/2021-07-11-16259930856620.jpg" alt="-w285"></li></ol><p>虽然玩法简单，但提供的数字却千变万化，所以很适合用程序来求解。</p><p>类似这种需要穷举的问题一般采用回溯法，也就是暴力求解，在最坏的情况下时间复杂度为指数。</p><h2 id="回溯法"><a href="#回溯法" class="headerlink" title="回溯法"></a>回溯法</h2><p>回溯法采用试错的思想，它尝试分步的去解决一个问题。在解决问题的过程中，如果现有的分步答案不能得到有效的正确的解答的时候，它将取消上一步甚至是上几步的计算，再通过其它的可能的分步解答再次尝试寻找问题的答案。</p><p>回溯法通常用最简单的递归方法来实现，下面是回溯法的一般结构。<br>理解回溯法首先要理解递归，理解递归的过程，以及递归的返回值。可以通过查看斐波那契数列的递归实现的调用栈来理解递归的过程。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">backtracking</span><span class="params">(args)</span>:</span></span><br><span class="line"> <span class="keyword">if</span> can_stop(args): <span class="comment"># 判断是否终止，限制了递归深度</span></span><br><span class="line"> <span class="keyword">if</span> need(): <span class="comment"># 必要时收集递归子树的叶子节点的结果</span></span><br><span class="line"> collect_result()</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> choice <span class="keyword">in</span> get_all_choices(args): <span class="comment"># 针对每一种情况, 情况的个数也就是每一层的广度</span></span><br><span class="line"> set_state(args, choice) <span class="comment"># 暂时选择该项</span></span><br><span class="line"> process(args) <span class="comment"># 该节点数据加工</span></span><br><span class="line"> backtracking(args) <span class="comment"># 递归进入下一层次</span></span><br><span class="line"> revert_state(args, choice) <span class="comment"># 撤销当前选择</span></span><br></pre></td></tr></table></figure><h2 id="暴力解法"><a href="#暴力解法" class="headerlink" title="暴力解法"></a>暴力解法</h2><p>回到数独解法上来</p><p>首先看最简单的暴力解法</p><p>这里有几个点和模板不一样</p><ul><li>由于把整个棋盘都填满游戏就终止了，所以不需要<code>can_stop</code>判断</li><li>每层递归会填满一个空位，递归的最大深度就是空位的个数。</li><li>数独棋盘都填满才算一个解，所以每层递归<code>get_all_choices</code>最多有 <code>行 x 列 x 数字取值 = 9 x 9 x 9</code> 种情况</li><li>由于数独规则的约束，行、列、九宫格内数字不能重复，所以天然有部分剪枝</li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 0 代表空位</span></span><br><span class="line">board = [</span><br><span class="line"> [<span class="number">0</span>, <span class="number">8</span>, <span class="number">6</span>, <span class="number">9</span>, <span class="number">0</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>],</span><br><span class="line"> [<span class="number">0</span>, <span class="number">0</span>, <span class="number">7</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>],</span><br><span class="line"> [<span class="number">5</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">7</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>],</span><br><span class="line"> [<span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">6</span>],</span><br><span class="line"> [<span class="number">0</span>, <span class="number">0</span>, <span class="number">5</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>],</span><br><span class="line"> [<span class="number">0</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">6</span>, <span class="number">0</span>, <span class="number">8</span>, <span class="number">2</span>],</span><br><span class="line"> [<span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">4</span>, <span class="number">0</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">5</span>, <span class="number">0</span>],</span><br><span class="line"> [<span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">0</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">3</span>],</span><br><span class="line"> [<span class="number">4</span>, <span class="number">1</span>, <span class="number">0</span>, <span class="number">6</span>, <span class="number">0</span>, <span class="number">5</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>],</span><br><span class="line">]</span><br><span class="line"></span><br><span class="line">EMPTY = <span class="number">0</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">sodoku</span><span class="params">(board)</span>:</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">can_put</span><span class="params">(board, row, col, c)</span>:</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> <span class="keyword">if</span> board[i][col] != EMPTY <span class="keyword">and</span> board[i][col] == c: <span class="comment"># 行不冲突</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> <span class="keyword">if</span> board[row][i] != EMPTY <span class="keyword">and</span> board[row][i] == c: <span class="comment"># 列不冲突</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> x, y = row // <span class="number">3</span> * <span class="number">3</span> + i // <span class="number">3</span>, col // <span class="number">3</span> * <span class="number">3</span> + i % <span class="number">3</span></span><br><span class="line"> <span class="keyword">if</span> board[x][y] != EMPTY <span class="keyword">and</span> board[x][y] == c: <span class="comment"># 九宫不冲突</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">True</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">solve</span><span class="params">(board)</span>:</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> <span class="keyword">if</span> board[i][j] == EMPTY:</span><br><span class="line"> <span class="keyword">for</span> c <span class="keyword">in</span> range(<span class="number">1</span>, <span class="number">10</span>):</span><br><span class="line"> <span class="keyword">if</span> can_put(board, i, j, c):</span><br><span class="line"> board[i][j] = c</span><br><span class="line"> <span class="keyword">if</span> solve(board):</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">True</span></span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> board[i][j] = EMPTY</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">True</span></span><br><span class="line"> <span class="keyword">return</span> solve(board)</span><br></pre></td></tr></table></figure><h2 id="剪枝"><a href="#剪枝" class="headerlink" title="剪枝"></a>剪枝</h2><p>所谓的剪枝，就是递归每一层的时候，并不是所有情况都是有效的，可以跳过这些分支。</p><p><img src="https://image.ponder.work/mweb/2021-07-11-16260036519609.jpg" alt="-w351"></p><p>以该题为例，第一行第三列可选取值并不是1到9，而是<code>1，2，4</code>，而且随着我们不断把空位填满，越后面的点选择越少。<br>可以用集合存储每个空位的行、列、九宫方向的可选值，三者交集就是该点的可选值。（用位替换集合还可以进一步优化算法）<br>因为填充一个空位，会影响该位置行、列、九宫上的所有空位的取值，所有不直接存储该点的可选值。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br></pre></td><td class="code"><pre><span class="line">EMPTY = <span class="number">0</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">sodoku</span><span class="params">(board)</span>:</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">solve</span><span class="params">(board)</span>:</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> box_index = (i // <span class="number">3</span> ) * <span class="number">3</span> + j // <span class="number">3</span></span><br><span class="line"> <span class="keyword">if</span> board[i][j] == EMPTY:</span><br><span class="line"> <span class="keyword">for</span> c <span class="keyword">in</span> (rows[i] & columns[j] & boxes[box_index]): <span class="comment"># 通过集合来剪枝</span></span><br><span class="line"> <span class="comment"># 设置状态</span></span><br><span class="line"> board[i][j] = c</span><br><span class="line"> tmp = [<span class="literal">False</span>, <span class="literal">False</span>, <span class="literal">False</span>]</span><br><span class="line"> <span class="keyword">for</span> idx, elem <span class="keyword">in</span> enumerate((rows[i], columns[j], boxes[box_index])):</span><br><span class="line"> <span class="keyword">if</span> c <span class="keyword">in</span> elem:</span><br><span class="line"> elem.remove(c)</span><br><span class="line"> tmp[idx] = <span class="literal">True</span></span><br><span class="line"> <span class="comment"># 进入下一层递归</span></span><br><span class="line"> <span class="keyword">if</span> solve(board):</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">True</span></span><br><span class="line"> <span class="comment"># 还原状态</span></span><br><span class="line"> board[i][j] = EMPTY</span><br><span class="line"> <span class="keyword">for</span> idx, elem <span class="keyword">in</span> zip(tmp, (rows[i], columns[j], boxes[box_index])):</span><br><span class="line"> <span class="keyword">if</span> idx:</span><br><span class="line"> elem.add(c)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">True</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 验证题目是否合法, 初始化每一格可选择项</span></span><br><span class="line"> rows = [set(range(<span class="number">1</span>,<span class="number">10</span>)) <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>)] <span class="comment"># 行内所有点的可选值</span></span><br><span class="line"> columns = [set(range(<span class="number">1</span>,<span class="number">10</span>)) <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>)] <span class="comment"># 列</span></span><br><span class="line"> boxes = [set(range(<span class="number">1</span>,<span class="number">10</span>)) <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>)] <span class="comment"># 九宫</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> range(<span class="number">9</span>):</span><br><span class="line"> num = board[i][j]</span><br><span class="line"> <span class="keyword">if</span> num != EMPTY:</span><br><span class="line"> num = int(num)</span><br><span class="line"> box_index = (i // <span class="number">3</span> ) * <span class="number">3</span> + j // <span class="number">3</span></span><br><span class="line"> <span class="keyword">if</span> num <span class="keyword">not</span> <span class="keyword">in</span> rows[i]:</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> rows[i].remove(num)</span><br><span class="line"> <span class="keyword">if</span> num <span class="keyword">not</span> <span class="keyword">in</span> columns[j]:</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> columns[j].remove(num)</span><br><span class="line"> <span class="keyword">if</span> num <span class="keyword">not</span> <span class="keyword">in</span> boxes[box_index]:</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"> boxes[box_index].remove(num)</span><br><span class="line"> <span class="keyword">return</span> solve(board)</span><br></pre></td></tr></table></figure><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><p><a href="https://zh.wikipedia.org/wiki/%E6%95%B8%E7%8D%A8" target="_blank" rel="noopener">https://zh.wikipedia.org/wiki/%E6%95%B8%E7%8D%A8</a><br><a href="https://zh.wikipedia.org/wiki/%E5%9B%9E%E6%BA%AF%E6%B3%95" target="_blank" rel="noopener">https://zh.wikipedia.org/wiki/%E5%9B%9E%E6%BA%AF%E6%B3%95</a><br><a href="https://www.jianshu.com/p/8e694d079a76" target="_blank" rel="noopener">https://www.jianshu.com/p/8e694d079a76</a></p>]]></content>
<summary type="html">
<p>数独是一种数学逻辑游戏，游戏由9×9个格子组成，玩家需要根据格子提供的数字推理出其他格子的数字。游戏设计者会提供最少17个数字使得解答谜题只有一个答案。</p>
<p>数独的解法需 遵循如下规则：</p>
<ol>
<li>数字 1-9 在每一行只能出现一次。</li>
<li>数字 1-9 在每一列只能出现一次。</li>
<li>数字 1-9 在每一个以粗实线分隔的 3x3 宫内只能出现一次。
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Python" scheme="http://ponder.work/tags/Python/"/>
<category term="算法" scheme="http://ponder.work/tags/%E7%AE%97%E6%B3%95/"/>
<category term="回溯法" scheme="http://ponder.work/tags/%E5%9B%9E%E6%BA%AF%E6%B3%95/"/>
</entry>
<entry>
<title>缓存淘汰算法之LFU</title>
<link href="http://ponder.work/2021/06/23/lfu-cache/"/>
<id>http://ponder.work/2021/06/23/lfu-cache/</id>
<published>2021-06-23T14:48:00.000Z</published>
<updated>2022-11-02T15:28:48.000Z</updated>
<content type="html"><![CDATA[<p>Least Frequently Used (LFU) 是一种常见的缓存淘汰算法，译为“最近最不经常使用”，也就是将缓存中使用次数最少的数据淘汰掉。</p><p>有两种常见的实现方法</p><ul><li>小顶堆 + hashmap，插入和删除的复杂度为O(logN), 但淘汰相同访问次数的节点是不稳定的，因为堆排序不稳定。</li><li>数组存储数据项 + hashmap记录数据项index, 淘汰缓存的复杂度为O(N)</li></ul><a id="more"></a><p>特点</p><ol><li>一般情况下，LFU效率要优于LRU，且能够避免周期性或者偶发性的操作导致缓存命中率下降的问题</li><li>LFU存在历史数据影响将来数据的”缓存污染”问题。</li></ol><h2 id="Python-实现"><a href="#Python-实现" class="headerlink" title="Python 实现"></a>Python 实现</h2><p>这里的Python实现是方案1</p><p>具体步骤</p><ol><li>get元素时，如果存在则返回结果并更新访问次数</li><li>set元素时，如果存在则更新val并更新访问次数，否则检查是否淘汰缓存并插入新key</li></ol><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> math <span class="keyword">import</span> log, ceil</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">MinHeap</span><span class="params">(object)</span>:</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__init__</span><span class="params">(self)</span>:</span></span><br><span class="line"> self._items = [<span class="literal">None</span>]</span><br><span class="line"> self.need_swap = self.more</span><br><span class="line"></span><br><span class="line"><span class="meta"> @property</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">length</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">return</span> len(self._items) - <span class="number">1</span></span><br><span class="line"></span><br><span class="line"><span class="meta"> @property</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">depth</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">return</span> ceil(log(self.length+<span class="number">1</span>, <span class="number">2</span>))</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">more</span><span class="params">(self, i, j)</span>:</span></span><br><span class="line"> <span class="keyword">return</span> self._items[i] > self._items[j]</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">exch</span><span class="params">(self, i, j)</span>:</span></span><br><span class="line"> self._items[i], self._items[j] = self._items[j], self._items[i]</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">swim</span><span class="params">(self, k)</span>:</span></span><br><span class="line"> <span class="keyword">while</span> k > <span class="number">1</span> <span class="keyword">and</span> self.need_swap(k//<span class="number">2</span>, k):</span><br><span class="line"> self.exch(k//<span class="number">2</span>, k)</span><br><span class="line"> k = k//<span class="number">2</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">sink</span><span class="params">(self, k)</span>:</span></span><br><span class="line"> <span class="keyword">while</span> <span class="number">2</span> * k <= self.length:</span><br><span class="line"> j = <span class="number">2</span> * k</span><br><span class="line"> <span class="keyword">if</span> j < self.length <span class="keyword">and</span> self.need_swap(j, j+<span class="number">1</span>):</span><br><span class="line"> j += <span class="number">1</span></span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> self.need_swap(k, j):</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> self.exch(k, j)</span><br><span class="line"> k = j</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">insert</span><span class="params">(self, val)</span>:</span></span><br><span class="line"> self._items.append(val)</span><br><span class="line"> self.swim(self.length)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">top</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">if</span> self.length > <span class="number">0</span>:</span><br><span class="line"> <span class="keyword">return</span> self._items[<span class="number">1</span>]</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">del_top</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">if</span> self.length > <span class="number">0</span>:</span><br><span class="line"> self.exch(<span class="number">1</span>, self.length)</span><br><span class="line"> val = self._items.pop()</span><br><span class="line"> self.sink(<span class="number">1</span>)</span><br><span class="line"> <span class="keyword">return</span> val</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__repr__</span><span class="params">(self)</span>:</span></span><br><span class="line"> tmp = []</span><br><span class="line"> seq = <span class="string">' '</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">1</span>, self.depth+<span class="number">1</span>):</span><br><span class="line"> l = seq.join([str(e) <span class="keyword">for</span> e <span class="keyword">in</span> self._items[<span class="number">2</span>**(i<span class="number">-1</span>):<span class="number">2</span>**i]])</span><br><span class="line"> tmp.append(l)</span><br><span class="line"> <span class="keyword">return</span> <span class="string">'\n'</span>.join(tmp)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Node</span>:</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__init__</span><span class="params">(self, key, val=None)</span>:</span></span><br><span class="line"> self.key = key</span><br><span class="line"> self.val = val</span><br><span class="line"> self.count = <span class="number">1</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__gt__</span><span class="params">(self, other)</span>:</span></span><br><span class="line"> <span class="keyword">return</span> self.count > other.count</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__repr__</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="comment"># return '<Node key={!r} val={!r} count={!r}>'.format(self.key, self.val, self.count)</span></span><br><span class="line"> <span class="keyword">return</span> <span class="string">'{}|{!r}'</span>.format(self.key, self.count)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">LFUCache</span>:</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__init__</span><span class="params">(self, size)</span>:</span></span><br><span class="line"> self.cache = {}</span><br><span class="line"> self.heap = MinHeap()</span><br><span class="line"> self.size = size</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">check_expired</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">if</span> self.heap.length == self.size:</span><br><span class="line"> node = self.heap.del_top()</span><br><span class="line"> self.cache.pop(node.key)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">update_count</span><span class="params">(self, node)</span>:</span></span><br><span class="line"> idx = self.heap._items.index(node)</span><br><span class="line"> node.count += <span class="number">1</span></span><br><span class="line"> self.heap.sink(idx)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">get</span><span class="params">(self, key)</span>:</span></span><br><span class="line"> node = self.cache.get(key, <span class="literal">None</span>)</span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> node:</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> self.update_count(node)</span><br><span class="line"> <span class="keyword">return</span> node.val</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">set</span><span class="params">(self, key, val)</span>:</span></span><br><span class="line"> node = self.cache.get(key, <span class="literal">None</span>)</span><br><span class="line"> <span class="keyword">if</span> node:</span><br><span class="line"> node.val = val</span><br><span class="line"> self.update_count(node)</span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"> node = Node(key, val)</span><br><span class="line"> self.check_expired()</span><br><span class="line"> self.heap.insert(node)</span><br><span class="line"> self.cache[key] = node</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__repr__</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">return</span> <span class="string">'<LFU {!r}>'</span>.format(self.cache)</span><br></pre></td></tr></table></figure><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://en.wikipedia.org/wiki/Least_frequently_used" target="_blank" rel="noopener">https://en.wikipedia.org/wiki/Least_frequently_used</a></li><li><a href="https://melonshell.github.io/2020/02/07/ds_cache_eli/" target="_blank" rel="noopener">https://melonshell.github.io/2020/02/07/ds_cache_eli/</a></li></ul>]]></content>
<summary type="html">
<p>Least Frequently Used (LFU) 是一种常见的缓存淘汰算法，译为“最近最不经常使用”，也就是将缓存中使用次数最少的数据淘汰掉。</p>
<p>有两种常见的实现方法</p>
<ul>
<li>小顶堆 + hashmap，插入和删除的复杂度为O(logN), 但淘汰相同访问次数的节点是不稳定的，因为堆排序不稳定。</li>
<li>数组存储数据项 + hashmap记录数据项index, 淘汰缓存的复杂度为O(N)</li>
</ul>
</summary>
<category term="编程" scheme="http://ponder.work/categories/%E7%BC%96%E7%A8%8B/"/>
<category term="Python" scheme="http://ponder.work/tags/Python/"/>
<category term="缓存" scheme="http://ponder.work/tags/%E7%BC%93%E5%AD%98/"/>
</entry>
<entry>
<title>学海屠魔录</title>
<link href="http://ponder.work/2021/06/20/learning-method/"/>
<id>http://ponder.work/2021/06/20/learning-method/</id>
<published>2021-06-20T03:17:00.000Z</published>
<updated>2021-07-15T08:02:44.000Z</updated>
<content type="html"><![CDATA[<p>英彦有云，魔鬼在细节之中（The devil is in the details），其意义在于提醒人们，不要忽视细节，即使是微不足道的小处也可以影响大局。</p><p>换个角度思考，这句话用在学习上，是再贴切不过了。当我们接触一个新领域时，其中的每个知识细节，都是魔鬼，稍有不慎就会被魔鬼击败，产生厌学情绪，再起不能。只有除尽这些魔鬼，才能攀上知识的高峰。<br><a id="more"></a></p><p>但光学会这些细节的点还远远不够，你的知识都是僵化的，是零散不成体系的的，难以真正应用。<br>就如将英语单词表背得滚瓜烂熟，倘若不识得语法与文法，断然是写不出文章的。</p><p>总地来说，学习分为两个阶段，“为学日益”和“为道日损”，也就是积累具体细节知识点，然后再将这些知识点进行归纳总结整理，将知识抽象并构建体系。</p><blockquote><p>注：此处的“为学日益”和“为道日损”与道德经中原义不同</p></blockquote><p>华罗庚教授曾把读书的过程归纳为“由薄到厚”与“由厚到薄”两个阶段，也是异曲同工；硅谷钢铁侠马斯克常说的“第一性原理”，就是第二步“为道日损”所得到的结果；再进一步，禅宗所说的“渐修”和“顿悟”也是一样的。</p><p>这两个阶段总体上是同等重要的，但在学习的不同阶段又该有所侧重。</p><p>下面将理论展开论述，我们又能引申出一些结论，得到一些新的看法。</p><h2 id="为学日益"><a href="#为学日益" class="headerlink" title="为学日益"></a>为学日益</h2><p>先讲第一阶段，这一阶段积累具体细节知识点，那么多大规模的知识点可以成为一个细节呢？</p><p>这点因人而异，因当前知识储备而异，评判标准就是能否说出该知识点的定义和用途，如若不能就应该继续往下看构成该知识点的子知识点，直至满足标准为止。</p><p>由此可知，学习是有极限的，当最小的知识点都不能领会时，继续学习就无从谈起了。<br>就如数学，当不能理解<code>1 + 1 = 2</code>时，在这之上构建的一切都与你无缘了。</p><p>所以学习还是要靠智商的，只不过要求极低。</p><p>在这个阶段比较有效的方法有</p><ul><li>不同体系的知识类比迁移，跟已学会的知识联系，便于记忆</li><li>费曼学习法，通过输出的方式，对学会的知识点的掌握程度进行确认。</li></ul><h2 id="为道日损"><a href="#为道日损" class="headerlink" title="为道日损"></a>为道日损</h2><p>当学习到达一定的阶段，就该对知识进行归纳整理，以便进入下一阶段的学习。</p><p>因为人脑所能同时关注的知识点是有上限的，如果不加以归纳整理进行抽象，体现出来就是学了新的忘了旧的。</p><p>日常生活的很多场景都与这个有关，比如人们在争辩的时候经常会扣帽子，这就是一种抽象，降低了受众的认知难度，在传播中是有利的。“xx人偷井盖”，这就是典型的扣帽子，一个地方人何其多，每个人又不一样，贴切地描述出这个整体是非常难的，扣帽子就是舍弃了其他特征，用某几个特征来概括整体，这样带了的坏处就是不准确，失之偏颇。</p><p>这一阶段影响的是对知识的应用，也就是将一系列知识点抽象成一个整体，用于构造其他知识。</p><p>抽象有好的有坏的，知识点的名称就是对其内涵的一种抽象，好的抽象应该是可以望文生义的。就像写代码，好的函数命名应该是表达了函数的用途，也就是对具体实现该函数语句的抽象。</p><p>类比也是一种抽象，而且是一种极其高效的抽象，将具体的细节和已有的知识点关联起来。但类比也是危险的，很容易做出不合适的类比，也就是引喻失义。所以类比更像是方便法门，一条通向“第一性原理”的捷径，有其适用场景，不应该作为“第一性原理”本身牢记。</p><h2 id="歧路"><a href="#歧路" class="headerlink" title="歧路"></a>歧路</h2><p>由此可以看出很多学习的方法其实都走在歧路上。</p><p>如学英语，一味在单词和例句中深挖，最终还是不能流畅的写作交流，这是第二阶段缺失造成的。<br>又如罗辑思维，听他灌输各种大而无当的大道理，只会显得很轻浮，除了增加谈资之外无有用处，这是第一阶段缺失造成的。</p><p>最后，希望我这胡言乱语对读者有些微帮助。</p>]]></content>
<summary type="html">
<p>英彦有云，魔鬼在细节之中（The devil is in the details），其意义在于提醒人们，不要忽视细节，即使是微不足道的小处也可以影响大局。</p>
<p>换个角度思考，这句话用在学习上，是再贴切不过了。当我们接触一个新领域时，其中的每个知识细节，都是魔鬼，稍有不慎就会被魔鬼击败，产生厌学情绪，再起不能。只有除尽这些魔鬼，才能攀上知识的高峰。<br>
</summary>
<category term="随笔" scheme="http://ponder.work/categories/%E9%9A%8F%E7%AC%94/"/>
<category term="随笔" scheme="http://ponder.work/tags/%E9%9A%8F%E7%AC%94/"/>
<category term="学习方法" scheme="http://ponder.work/tags/%E5%AD%A6%E4%B9%A0%E6%96%B9%E6%B3%95/"/>
</entry>
</feed>