[colug-432] 2014-09-24會議 Scribbles 落書/惡文?
jep200404 at columbus.rr.com
jep200404 at columbus.rr.com
Thu Sep 25 21:23:58 EDT 2014
On Wed, 03 Sep 2014 20:54:48 -0400, Scott Merrill <skippy at skippy.net> wrote:
> Jim Prior will present on editing the command line and scripting
> on the command line. Jim will show one or more ways of doing
> something, and then others will share better ways of do that same
> thing. This should be an interactive and pretty practical
> meeting.
>
> for f in *.htm; do echo mv -i "$f" "${f%%.htm}.html";done
> for f in index.*.html; do echo mv -i "$f" "ndx.${f##index.}";done
Release early, release often.
Fun was had playing around with the command line.
This presentation was inspired by mailing list thread[1], which said in part:
> I am no longer shocked at how many do not understand the
> basic productivity enhancement tools available to them like
> shell history
> screen
> commands like watch
> in place shell programming (basic for loops, etc)
and
> And _most_ only know about the arrow keys for history.
Of those, this presentation was about:
o searching the history
o editing the command line
o loops
o some long strings of programs piped together
First a little presentation about giving presentations.
The key thing is that the presentation should be easily viewed by everyone
in the audience, including those in the back row.
Resolution
24 * 80 is about max resolution
Colors
Room lights on projection screens is common and bad, so maximize
the contrast in your content to salvage what contrast is available
to make it easier to see.
GOOD:
white on black
black on white
TOLERABLE:
pastels on white
BAD, VERY VERY BAD:
colorized text on black
combining multiple commands, especially loops
In the following "stgfopl" means something that generates filenames, one per
line. Examples could include:
ls *
ls *.JPG
find . -type f -name '*.pdf' -print
loops
Here are some common patterns I use:
for i in `seq 1234 4566`;do echo do something with "$i";done
for f in *.pdf; do echo do something with "$f";done
stgfopl | while read f; do echo do something with "$f";done
stgfopl | xargs some command
Most of the rest of this presentation will play with examples of those
common patterns.
I used to have loops like:
i=5;while [ $i -le 10 ]; do echo $i;i=`expr $i + 1`;done
I now usually use seq instead:
for i in `seq 5 10`;do echo $i;done
The former style still has a place where the output of seq is too much
for the shell, of if the iteration is not just simple arithmetic
sequence. For example, consider the following:
i=1;while [ $i -le `echo '2^32' | bc` ]; do echo $i;i=`expr $i + $i`;done
or for following links, such as in a chain of soft links, or a
linked listg.
find . -name '*.pdf' -print | xargs wc
find . -name '*.pdf' -print | while read f; do echo do something with "$f";done
for i in `seq 2622 2635`; do mv -i /media/jep/disk/DCIM/100GEDSC/GEDC`printf '%04d' "$i"`.* .;done
for i in `seq 2622 2635`; do f=GEDC`printf '%04d' "$i"`.JPG;mv -i "$f" "${f%%.JPG}.jpeg" ;done
Study, compare, and contrast:
"${f%%.JPG}"
"${f%.JPG}"
"${f##hello.}"
"${f#hello}"
Some loops go on forever:
while true; do time factor 1111111111111111111;done #32-bit
while true; do time factor 11111111111111111111111111111111111111;done #64-bit
while true; do date | tr '\n' '\r';sleep .1;done
watch -n .1 "date | tr '\n' '\r'"
history | sed -e 's/^ *[0-9][0-9]* [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9] //gi' | sort | uniq -c | sort -nr | head
Use echo to see what commands would be like (i.e., dry run) before actually
doing commands.
for f in *.JPG; do echo mv -i "$f" "${f%%.JPG}.jpeg" ;done
for f in *.JPG; do mv -i "$f" "${f%%.JPG}.jpeg" ;done
classic pipelines
# Find the most common commands.
# Often there are multiple correct ways to do something.
history | awk '{print $4}' | sort | uniq -c | sort -n | tail
history | awk '{print $4}' | sort | uniq -c | sort -n | tac | head
history | awk '{print $4}' | sort | uniq -c | sort -nr | head
but the focus of this presentation is more complicated stuff,
particularly loops.
How long of a line is too long?
If it makes your head hurt, it's too long.
If it slows you down, it's too long.
Different people tolerate different amounts of complexity.
If you're repeating some monster command often,
it might be a good candidate to put into a shell script.
history options
The history can have timestamps. This can help one look for an old command that
one ran some time ago, by narrowing how much of the history one looks at. When
developing scripts, I will often paste multiple lines from them at once. The
command timestamps can document how fast things worked. Hence the following
line in .bashrc.
HISTTIMEFORMAT="%F %T "
# Comment commands
Temporarily parking a command
In the middle of writing some command, I realize that I need to run
some other command first, so I put '#' at beginning of line and execute
it, so the history remembers the command I was working on. Then run the
command I needed to do first, then go back in recent history to get the
abandoned command, delete the leading # and execute it.
to document the history
Russ mentioned deliberating executing a comment command, so that it
ends up in the history to document the following (or preceding)
commands.
new bash exploit
http://lists.colug.net/pipermail/colug-432/2014-September/003356.html
http://fedoramagazine.org/flaw-discovered-in-the-bash-shell-update-your-fedora-systems/
http://www.zdnet.com/unixlinux-bash-critical-security-hole-uncovered-7000034021/
alias lll='ls -l --full-time'
The --full-time output is easier for downstream programs to sort by date
and/or time. Avoids difficulty of sorting month abbreviations. Also, the
format does not switch for files more than a year old (time of day versus
year). Compare the output of ll versus lll:
jep at presentation:~$ ll -tr /etc
total 1276
-rw-r--r-- 1 root root 2064 Nov 23 2006 netscsid.conf
...
-rw-r--r-- 1 root root 1189 Sep 25 20:20 mtab
jep at presentation:~$ lll -tr /etc
total 1276
-rw-r--r-- 1 root root 2064 2006-11-23 14:33:10.000000000 -0500 netscsid.conf
...
-rw-r--r-- 1 root root 1189 2014-09-25 20:20:46.194351828 -0400 mtab
jep at presentation:~$
some nifty things are available in bash, but not Bourne shell
avoiding them is good for portability
source instead of . (from csh, thanks Rob)
$(command) instead of `command`
$(command) can be nested.
`command` can not be nested
>(command)
... | tee "$f" | tee >(md5sum) >(sha1sum) >(sha256sum) /dev/null
... | tee "$f" >(gzip >"$f".gz) >(bzip2 >"$f".bz2) /dev/null
drive=/dev/sr0;readrawcd $drive | tee >(md5sum) >(sha1sum) >(sha256sum) >/dev/null;sleep 10;eject $drive;date
drive=xubuntu-12.04.4-desktop-i386.iso;readrawcd $drive | tee >(md5sum) >(sha1sum) >(sha256sum) >/dev/null;sleep 10;eject $drive;date
tees tease: for fun try the above tees without the /dev/null
<(command)
vidf old new
jep at presentation:~$ cat ~/bin/vidf
# vim -O "$1" "$2" <(diff "$1" "$2")
vim -O <(diff "$1" "$2") "$1" "$2"
jep at presentation:~$
$((arithmetic expression)) (e.g., $(($i + 1))
i=5
echo `expr $i + 1`
echo `echo "$i + 1" | bc`
echo `echo "print $i + 1" | python`
# How to do above with awk instead of python?
Miscellaneous
Begin dangerous commands with '#' or echo while editing them.
Remove the '#' or echo after one is satisfied that the command is correct.
# rm -rf / foo
# rm -rf /foo
rm -rf /foo
echo rm -rf / foo
echo rm -rf /foo
rm -rf /foo
sudo !! is handy for rerunning command with sudo
rm -rf /tmp/*
sudo !!
cmatrix text "screensave"
On a sacrificial Ubuntu box in an xterm:
sudo apt-get update
sudo apt-get install cmatrix
cmatrix
.bashrc
Many programs pay attention to the following:
EDITOR=vim
export EDITOR
history options: I commented out some of Xubuntu's defaults.
# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
#HISTCONTROL=ignoreboth
# append to the history file, don't overwrite it
shopt -s histappend
Jason and Russ, please elaborate on the benefits and drawbacks of
"shopt -s histappend".
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=400000
HISTFILESIZE=400000
# HISTCONTROL=ignoredups:ignorespace
I want all commands in the history. I _do_ want to save the
duplicate commands. Sometimes repeated commands are necessary for
some task. Having the history remember the repeated commands is
helpful for repeating the whole sequence again later. Sometimes I
want to see how long each command took. Timestamps for repeated
commands answer that. My command to show the most common commands
relies upon those duplicate commands being in the
history.
unset HISTCONTROL
HISTFILESIZE=400000
HISTSIZE=400000
HISTTIMEFORMAT="%F %T "
export HISTCONTROL
export HISTFILESIZE
export HISTSIZE
EMACS is default for editing command line and history.
set -o vi to select vi for editing command line
git stuff
I need to use git version control more. Good candidates are:
gas price stuff
.bashrc
.vimrc
vidf
I found http://www.git-scm.com/book/en/Git-on-the-Server and other git
tutorials on that site to be helpful. Rebasing is cool.
locate foo | xargs md5sum | sort
Russ, what the line number and URL for your nasty example from
https://github.com/herrold/?
Is Apple's default search engine now duckduckgo, not Google anymore?
Has Apple's canary disappeared?
tmux
Jason, please elaborate on virtue of using ^Q.
tmux pipe-pane -o 'cat >>'$PWD/tmux.output.`date +%Y%m%d-%H%M%S`
Screenkey was used to show what keys were typed to edit the command line and
history. Need to tweak screenkey to:
put it out of the way to waste less space on screen
make it smaller
make it persist (a log file would be nice too)
https://launchpad.net/screenkey????
I would be happy with key logger output showing in an xterm window.
# Redirect stdout and stderr with timestamping to log file.
exec 1> >(timestamper | tee -a log) 2>&1
jep at presentation:~$ cat /home/jep/tools/timestamper.c
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <time.h>
#include <sys/time.h>
#include <string.h>
#define False (0)
#define True (!False)
typedef unsigned char flag;
#define ArrayLength(x) (sizeof(x)/sizeof(*(x)))
void print_timestamp(void)
{
char buf[80];
int i;
struct timeval now;
struct tm *b;
gettimeofday(&now,NULL);
b=localtime(&now.tv_sec);
i=sprintf(buf,"%04d-%02d-%02d %02d:%02d:%02d.%06d "
, b->tm_year+1900
, b->tm_mon+1
, b->tm_mday
, b->tm_hour
, b->tm_min
, b->tm_sec
, (unsigned int)now.tv_usec);
write(STDOUT_FILENO,buf,i);
}
int main(int argc,char *argv[])
{
unsigned char buf[1];
flag please_print_timestamp=True;
int i;
while ((i=read(STDIN_FILENO,buf,ArrayLength(buf)))>0) {
if (please_print_timestamp) {
please_print_timestamp=False;
print_timestamp();
}
write(STDOUT_FILENO,buf,i);
if (buf[0]=='\n')
please_print_timestamp=True;
}
return EXIT_SUCCESS;
}
jep at presentation:~/monarch/dev-code$
Impedance matching:
Cracking a whip reminds me of how tsunamis can travel in fast, short, long
waves over vast stretches of deep water, then become tall waves in short
water. What's the electrical analogy to cracking whips and tsunamis? It's
probably a bunch of capacitors dumping into some tuned LC network. Hmmm.
Study microwave strip stuff. It also reminds me of Francis turbines.
wp:is a prefix for wikipedia
wp:Gödel, Escher, Bach
[1] It got interesting, starting around here:
http://lists.colug.net/pipermail/colug-432/2014-July/003223.html
http://lists.colug.net/pipermail/colug-432/2014-July/003225.html
http://lists.colug.net/pipermail/colug-432/2014-July/thread.html#3223
More information about the colug-432
mailing list