-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathtips.txt
238 lines (187 loc) · 8.66 KB
/
tips.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
# 20190319
# find symbol in workspace: VSCode
# type '#' in command pallete
# see also: https://code.visualstudio.com/docs/editor/editingevolved
# 20190316
# installing node.js on MacOS via HomeBrew:
https://www.dyclassroom.com/howto-mac/how-to-install-nodejs-and-npm-on-mac-using-homebrew
# default usergroup on MacOS is 'staff':
https://superuser.com/questions/590683/how-do-i-find-my-user-id-and-group-in-mac-os-x
# ref:
https://github.com/Homebrew/brew/issues/276#issuecomment-221692216
========================================================================================
# https://stackoverflow.com/questions/651018/opening-a-tar-gz-file-with-a-single-command
tar xzf file.tar.gz
x - extract
z - gunzip the input
f - Read from a file, not stdin
# remote-sync
rsync -r ~/Desktop/style-test/ wnixalo@WNX-G750JS:~/Kaukasos/FAI02/data/lesson9/style-test/
rsync -r ~/FROM_HERE/ wnixalo@WNX-G750JS:~/TO_HERE/
# JupyterNotebook SSH
(host): jupyter notebook --no-browser --port=8889
(client): ssh -NL 8889:localhost:8889 wnixalo@WNX-G750JS
# On ssh broken-pipes (client/server intervals):
- https://unix.stackexchange.com/questions/2010/what-does-the-broken-pipe-message-mean-in-an-ssh-session
-https://unix.stackexchange.com/questions/259225/packet-write-wait-broken-pipe-even-leaving-top-running
- https://coderwall.com/p/8ag5aq/ssh-broken-pipe-fix-mac-os-x
- https://askubuntu.com/questions/127369/how-to-prevent-write-failed-broken-pipe-on-ssh-connection
- https://forums.aws.amazon.com/thread.jspa?threadID=228154
- https://unix.stackexchange.com/questions/3026/what-options-serveraliveinterval-and-clientaliveinterval-in-sshd-config-exac
- http://forums.fast.ai/t/keep-ssh-connection-alive/2849/14
- https://thecustomizewindows.com/2017/06/fix-write-failed-broken-pipe-packet_write_wait-connection-port-22-broken-pipe/
# git credential store:
https://git-scm.com/docs/git-credential-store
# add ipykernel:
http://ipython.readthedocs.io/en/stable/install/kernel_install.html
source activate other-env
python -m ipykernel install --user --name other-env --display-name "Python (other-env)"
# remove ipykernel:
https://stackoverflow.com/questions/42635310/remove-kernel-on-jupyter-notebook
jupyter kernelspec list
# bash shell script:
https://askubuntu.com/questions/223691/how-do-i-create-a-script-file-for-terminal-commands/223698
https://stackoverflow.com/questions/22656359/bash-shell-script-opening-multiple-terminals-and-executing-distinct-commands
# Linux/UNIX: check drive usage via terminal:
- https://askubuntu.com/questions/432836/how-can-i-check-disk-space-used-in-a-partition-using-the-terminal-in-ubuntu-12-0/432842
- https://askubuntu.com/questions/73160/how-do-i-find-the-amount-of-free-space-on-my-hard-drive
df -h --total
# can also get pydf: sudo apt-get install pydf
# Find where a Python module is imported from:
https://stackoverflow.com/questions/7150998/where-is-module-being-imported-from
>>> import my_module
>>> my_module.__file__
# Install a local package w/ a Makefile directly to Conda Env (after $ make):
https://docs.python.org/2/install/index.html#alternate-installation-unix-the-prefix-scheme
python setup.py install --prefix=<conda env path>
# NOTE: must activate env first
# ImportError: numpy.core.multiarray failed to import:
https://github.com/pytorch/pytorch/issues/2731
>>> pip install numpy -I
# ssh AWS
$ ssh -i .ssh/ssh-key.pem remoteuser@remoteaddress
#
# Example: ssh -i .ssh/fastai_wnx_3.pem [email protected] -L8888:localhost:8888
# See: http://forums.fast.ai/t/great-summary-of-how-to-use-aws/7651
# ssh tunnel (Jupyter) AWS
$ ssh -i .ssh/ssh-key.pem remoteuser@remoteaddress -L portnum:localhost:portnum
## vnc screen sharing - linux remote desktop
# setup on linux host:
sudo apt-get install xorg lxde-core tightvncserver firefox lxterminal xfonts-100dpi
cat >> ~/.vnc/xstartup
lxterminal &
/usr/bin/lxsession -s LXDE &
<ctrl-d>
# turning on:
tightvncserver :13 -geometry 1200x900
#turning off:
tightvncserver -kill :13
## vnc screen sharing - connecting from Mac
#config
System Preferences -> Sharing: [check] Screen Sharing
#connect
(ssh into linux host -- use [-L59XX:localhost:59XX] where XX any 2 numbers (here: 13)
(open) Screen Sharing
localhost:5913
# rsync AWS
http://www.anthonychambers.co.uk/blog/rsync-to-aws-ec2-using-.pem-key/9
rsync -rave "ssh -i $SSHKEYPATH" REMOTENAME@REMOTEIP:PATHTOFILE $SAVETOPATH
# rsync AWS Spot:
$ rsync -rav ~/Downloads/kaggle.json "ssh -i ~/.ssh/id_rsa.pub" [email protected]:~/
$ rsync -rav $FROMHERE $TOHERE
# finding offline documentation for ReadTheDocs:
http://natanyellin.com/2012/04/22/downloading-pdf-documentation-for-readthedocs-projects/
# Atom jump to definition - symbols
https://stackoverflow.com/a/45402968
https://atom.io/packages/atom-ide-ui
cmd-click : go to definition (anywhere)
ctrl-cmd-G : select all instances in file for editing
cmd-click : multiple text cursors on click
option-shift-cmd-F : find references
cmd-option (hold) : show args
# summarize disk size of all items in current directory:
du -h --max-depth=1
# graphviz error:
https://stackoverflow.com/questions/35064304/runtimeerror-make-sure-the-graphviz-executables-are-on-your-systems-path-aft
# resetting git repo fork to original repo:
# https://stackoverflow.com/a/9646323s
git remote add upstream /url/to/original/repo
git fetch upstream
git checkout master
git reset --hard upstream/master
git push origin master --force
# get a cool blue box in MarkDown & Jupyter Notebooks:
<div class="alert alert-info">
...text...
</div>
## Jupyter Hub:
#re-sync a fork with the original repo:
https://stackoverflow.com/a/7244456
Installation of Jupyterhub on remote server:
https://github.com/jupyterhub/jupyterhub/wiki/Installation-of-Jupyterhub-on-remote-server
JupyterHub Docs: Security Settings:
https://jupyterhub.readthedocs.io/en/latest/getting-started/security-basics.html#enabling-ssl-encryption
(YouTube) JupyterHub: Deploying Jupyter Notebooks
https://www.youtube.com/watch?v=gSVvxOchT8Y
## dark jupyter notebook -- custom.css
https://www.reddit.com/r/Python/comments/4cbxo7/dark_theme_for_jupyteripython_notebooks/
## finding colors:
# hex code inspector:
https://www.color-hex.com
# rgba insepctor:
http://colorizer.org
# macos: digital color meter
## printing to console:
# use fastprogress:
https://github.com/fastai/fastprogress
# if redirecting to file, manually print via:
print(f'\r{yourtext}', end='\r') ## https://stackoverflow.com/a/34325723
## printing (ANSI) colors to console:
https://stackoverflow.com/a/287944
## git tips -- cancelling a commit
https://sethrobertson.github.io/GitFixUm/fixup.html#remove_last
git reset HEAD^
or: git reset HEAD-2
## transferring files from gcp storage to instance:
gsutil cp -r gs://[bucket name] [directory in VM]
# https://groups.google.com/d/msg/gce-discussion/JmW2nhQsMw0/ZWrUzO3nAgAJ
## enable conda env integration w/ IPython kernels (Jupyter)
conda install --channel=conda-forge nb_conda_kernels
- https://github.com/jupyter/jupyter/issues/245#issuecomment-287080203
# this may also be the way:
- https://stackoverflow.com/a/48349338
# perhaps also this:
- https://github.com/jupyter/jupyter/issues/245#issuecomment-287080203
## for-loops in bash shell
# https://www.cyberciti.biz/faq/bash-for-loop/
## instal .deb file from linux terminal:
# https://unix.stackexchange.com/a/159114
## docs on git stash
# https://git-scm.com/docs/git-stash
## gist on git upstream
# https://gist.github.com/CristinaSolana/1885435
## git config
# https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration
## git set global username and password (unencrypted)
# https://stackoverflow.com/questions/11403407/git-asks-for-username-every-time-i-push
git config --global user.name USERNAME <--- seems to work but not work
git config --global user.password PASSWORD <--- seems to work but not work
# and/or:
git config credential.helper store
# then enter 1 more time on push
## see conda environments in jupyter
# https://dev.widemeadows.de/2017/08/23/use-your-conda-environment-in-jupyter-notebooks/
conda install nb_conda
## jupyter full-width cells:
# https://www.reddit.com/r/IPython/comments/27zash/can_i_increase_notebook_cell_width_on_wide_screens/ci5zl6y
# https://github.com/jupyter/notebook/issues/1909#issuecomment-266116532
## release GPU memory after CUDA out of memory error:
# delete learner / model
gc.collect()
# see: https://forums.fast.ai/t/unet-learer-cuda-error/32592/8?u=borz
## start GCP instance:
gcloud compute ssh --zone=$ZONE jupyter@fastai-instance -- -L 8080:localhost:8080
# where $ZONE=us-east4-a
## specifying vertical ruler in vscode:
cmd-shift-p : settings --> 'editior.rulers' --> enter int/s in brackets, on right
# https://stackoverflow.com/questions/29968499/vertical-rulers-in-visual-studio-code