[colug-432] 2014-09-24會議 Scribbles 落書/惡文?

jep200404 at columbus.rr.com jep200404 at columbus.rr.com
Thu Sep 25 21:23:58 EDT 2014

On Wed, 03 Sep 2014 20:54:48 -0400, Scott Merrill <skippy at skippy.net> wrote:

> Jim Prior will present on editing the command line and scripting
> on the command line. Jim will show one or more ways of doing
> something, and then others will share better ways of do that same
> thing. This should be an interactive and pretty practical
> meeting.
>     for f in *.htm; do echo mv -i "$f" "${f%%.htm}.html";done
>     for f in index.*.html; do echo mv -i "$f" "ndx.${f##index.}";done

Release early, release often.
Fun was had playing around with the command line.

This presentation was inspired by mailing list thread[1], which said in part:

    > I am no longer shocked at how many do not understand the
    > basic productivity enhancement tools available to them like

    > shell history 
    > screen
    > commands like watch
    > in place shell programming (basic for loops, etc)


    > And _most_ only know about the arrow keys for history.

Of those, this presentation was about:
    o   searching the history
    o   editing the command line 
        o   loops
        o   some long strings of programs piped together

First a little presentation about giving presentations.

    The key thing is that the presentation should be easily viewed by everyone
    in the audience, including those in the back row.


            24 * 80 is about max resolution


            Room lights on projection screens is common and bad, so maximize
            the contrast in your content to salvage what contrast is available
            to make it easier to see.


                white on black
                black on white


                pastels on white

            BAD, VERY VERY BAD:

                colorized text on black

combining multiple commands, especially loops
In the following "stgfopl" means something that generates filenames, one per
line. Examples could include: 

    ls *
    ls *.JPG
    find . -type f -name '*.pdf' -print


        Here are some common patterns I use:

            for i in `seq 1234 4566`;do echo do something with "$i";done
            for f in *.pdf; do echo do something with "$f";done
            stgfopl | while read f; do echo do something with "$f";done
            stgfopl | xargs some command

        Most of the rest of this presentation will play with examples of those
        common patterns.

        I used to have loops like: 

            i=5;while [ $i -le 10 ]; do echo $i;i=`expr $i + 1`;done

        I now usually use seq instead:

            for i in `seq 5 10`;do echo $i;done

        The former style still has a place where the output of seq is too much
        for the shell, of if the iteration is not just simple arithmetic
        sequence. For example, consider the following:

            i=1;while [ $i -le `echo '2^32' | bc` ]; do echo $i;i=`expr $i + $i`;done
        or for following links, such as in a chain of soft links, or a 
        linked listg.

        find . -name '*.pdf' -print | xargs wc
        find . -name '*.pdf' -print | while read f; do echo do something with "$f";done

        for i in `seq 2622 2635`; do mv -i /media/jep/disk/DCIM/100GEDSC/GEDC`printf '%04d' "$i"`.* .;done
        for i in `seq 2622 2635`; do f=GEDC`printf '%04d' "$i"`.JPG;mv -i "$f" "${f%%.JPG}.jpeg" ;done

            Study, compare, and contrast:


        Some loops go on forever:

            while true; do time factor 1111111111111111111;done #32-bit
            while true; do time factor 11111111111111111111111111111111111111;done #64-bit
            while true; do date | tr '\n' '\r';sleep .1;done
            watch -n .1 "date | tr '\n' '\r'"

        history | sed -e 's/^ *[0-9][0-9]*  [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9] //gi' | sort | uniq -c | sort -nr | head

    Use echo to see what commands would be like (i.e., dry run) before actually
    doing commands.

        for f in *.JPG; do echo mv -i "$f" "${f%%.JPG}.jpeg" ;done
        for f in *.JPG; do      mv -i "$f" "${f%%.JPG}.jpeg" ;done

    classic pipelines

        # Find the most common commands.
        # Often there are multiple correct ways to do something.
        history | awk '{print $4}' | sort | uniq -c | sort -n | tail
        history | awk '{print $4}' | sort | uniq -c | sort -n | tac | head
        history | awk '{print $4}' | sort | uniq -c | sort -nr | head

        but the focus of this presentation is more complicated stuff,
        particularly loops.

How long of a line is too long? 

    If it makes your head hurt, it's too long. 
    If it slows you down, it's too long.
    Different people tolerate different amounts of complexity.
    If you're repeating some monster command often,
    it might be a good candidate to put into a shell script.

history options

The history can have timestamps. This can help one look for an old command that
one ran some time ago, by narrowing how much of the history one looks at. When
developing scripts, I will often paste multiple lines from them at once. The
command timestamps can document how fast things worked. Hence the following
line in .bashrc. 

# Comment commands

    Temporarily parking a command

        In the middle of writing some command, I realize that I need to run
        some other command first, so I put '#' at beginning of line and execute
        it, so the history remembers the command I was working on. Then run the
        command I needed to do first, then go back in recent history to get the
        abandoned command, delete the leading # and execute it.

    to document the history

        Russ mentioned deliberating executing a comment command, so that it
        ends up in the history to document the following (or preceding)

new bash exploit


alias lll='ls -l --full-time'

    The --full-time output is easier for downstream programs to sort by date
    and/or time. Avoids difficulty of sorting month abbreviations. Also, the
    format does not switch for files more than a year old (time of day versus
    year). Compare the output of ll versus lll:

    jep at presentation:~$ ll -tr /etc
    total 1276
    -rw-r--r--  1 root root     2064 Nov 23  2006 netscsid.conf
    -rw-r--r--  1 root root     1189 Sep 25 20:20 mtab
    jep at presentation:~$ lll -tr /etc
    total 1276
    -rw-r--r--  1 root root     2064 2006-11-23 14:33:10.000000000 -0500 netscsid.conf
    -rw-r--r--  1 root root     1189 2014-09-25 20:20:46.194351828 -0400 mtab
    jep at presentation:~$ 

some nifty things are available in bash, but not Bourne shell
avoiding them is good for portability

    source instead of . (from csh, thanks Rob)

    $(command) instead of `command`

        $(command) can be nested.
        `command` can not be nested


        ... | tee "$f" | tee >(md5sum) >(sha1sum) >(sha256sum) /dev/null 
        ... | tee "$f" >(gzip >"$f".gz) >(bzip2 >"$f".bz2) /dev/null 
        drive=/dev/sr0;readrawcd $drive | tee >(md5sum) >(sha1sum) >(sha256sum) >/dev/null;sleep 10;eject $drive;date
        drive=xubuntu-12.04.4-desktop-i386.iso;readrawcd $drive | tee >(md5sum) >(sha1sum) >(sha256sum) >/dev/null;sleep 10;eject $drive;date

        tees tease: for fun try the above tees without the /dev/null


        vidf old new

        jep at presentation:~$ cat ~/bin/vidf
        # vim -O "$1" "$2" <(diff "$1" "$2")
        vim -O <(diff "$1" "$2") "$1" "$2"
        jep at presentation:~$ 

    $((arithmetic expression)) (e.g., $(($i + 1))

        echo `expr $i + 1`
        echo `echo "$i + 1" | bc`
        echo `echo "print $i + 1" | python`
        # How to do above with awk instead of python?


    Begin dangerous commands with '#' or echo while editing them.
    Remove the '#' or echo after one is satisfied that the command is correct.

        # rm -rf / foo
        # rm -rf /foo
        rm -rf /foo

        echo rm -rf / foo
        echo rm -rf /foo
        rm -rf /foo

    sudo !! is handy for rerunning command with sudo

        rm -rf /tmp/*
        sudo !!

    cmatrix text "screensave"

        On a sacrificial Ubuntu box in an xterm:

            sudo apt-get update
            sudo apt-get install cmatrix


    Many programs pay attention to the following:

        export EDITOR

    history options: I commented out some of Xubuntu's defaults.

        # don't put duplicate lines or lines starting with space in the history.
        # See bash(1) for more options

        # append to the history file, don't overwrite it
        shopt -s histappend

            Jason and Russ, please elaborate on the benefits and drawbacks of
            "shopt -s histappend". 

        # for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
        # HISTCONTROL=ignoredups:ignorespace

            I want all commands in the history. I _do_ want to save the
            duplicate commands. Sometimes repeated commands are necessary for
            some task. Having the history remember the repeated commands is
            helpful for repeating the whole sequence again later. Sometimes I
            want to see how long each command took. Timestamps for repeated
            commands answer that. My command to show the most common commands
            relies upon those duplicate commands being in the

        unset HISTCONTROL
        export HISTCONTROL
        export HISTFILESIZE
        export HISTSIZE

    EMACS is default for editing command line and history.
    set -o vi to select vi for editing command line

git stuff

    I need to use git version control more. Good candidates are:

        gas price stuff

    I found http://www.git-scm.com/book/en/Git-on-the-Server and other git
    tutorials on that site to be helpful. Rebasing is cool. 

locate foo | xargs md5sum | sort

Russ, what the line number and URL for your nasty example from

Is Apple's default search engine now duckduckgo, not Google anymore? 
Has Apple's canary disappeared?


    Jason, please elaborate on virtue of using ^Q.

    tmux pipe-pane -o 'cat >>'$PWD/tmux.output.`date +%Y%m%d-%H%M%S`

Screenkey was used to show what keys were typed to edit the command line and
history. Need to tweak screenkey to:

    put it out of the way to waste less space on screen
    make it smaller
    make it persist (a log file would be nice too)


    I would be happy with key logger output showing in an xterm window.

# Redirect stdout and stderr with timestamping to log file.
exec 1> >(timestamper | tee -a log) 2>&1

jep at presentation:~$ cat /home/jep/tools/timestamper.c
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <time.h>
#include <sys/time.h>
#include <string.h>

#define False (0)
#define True (!False)

typedef unsigned char flag;

#define ArrayLength(x) (sizeof(x)/sizeof(*(x)))

void print_timestamp(void)
    char buf[80];
    int i;
    struct timeval now;
    struct tm *b;

    i=sprintf(buf,"%04d-%02d-%02d %02d:%02d:%02d.%06d "
    , b->tm_year+1900
    , b->tm_mon+1
    , b->tm_mday
    , b->tm_hour
    , b->tm_min
    , b->tm_sec
    , (unsigned int)now.tv_usec);


int main(int argc,char *argv[])
    unsigned char buf[1];
    flag please_print_timestamp=True;
    int i;

    while ((i=read(STDIN_FILENO,buf,ArrayLength(buf)))>0) {
        if (please_print_timestamp) {
        if (buf[0]=='\n')
    return EXIT_SUCCESS;
jep at presentation:~/monarch/dev-code$ 

Impedance matching:

    Cracking a whip reminds me of how tsunamis can travel in fast, short, long
    waves over vast stretches of deep water, then become tall waves in short
    water. What's the electrical analogy to cracking whips and tsunamis? It's
    probably a bunch of capacitors dumping into some tuned LC network. Hmmm.
    Study microwave strip stuff. It also reminds me of Francis turbines.

wp:is a prefix for wikipedia
wp:Gödel, Escher, Bach

[1] It got interesting, starting around here:

More information about the colug-432 mailing list