wikipedia article for details. Under (at least) some BSD-like systems and OS X the analogous command is chflags. The append-only feature is split between "user" and "system" versions presumably so non-root users can use it on their own files. Linux appears to have no equivalent way for a non-root user to set/unset append-only.

Prevent mangled history with atomic writes and lock files

TODO...

Compressing History Files

The result of the above is a history file with a great many duplicate commands. Appending history causes your history file to grow by all the shell's loaded history each time.

More importantly, the main thing we care about with regards to history is being able find previously executed commands. The following script will remove all commands from the history file that are already in there, while keeping the order of the commands intact in such a way that commands you most recently executed will remain at the bottom of the file (ie. keep the last occurrence of a command, not the first).

    awk 'NR==FNR && !/^#/{lines[$0]=FNR;next} lines[$0]==FNR' "$HISTFILE" "$HISTFILE" > "$HISTFILE.compressed" &&
    mv "$HISTFILE.compressed" "$HISTFILE"

After a few months, this compressed my history file from 761474 lines to 2349. Note that this does not preserve the timestamps if you have HISTTIMEFORMAT set.

Archiving History Files

Once you have enabled these methods, you should find that your bash history becomes much more valuable, allowing you to recall any command you have executed at any time. As such, you should ensure your history file(s) are included in your regular backups.

You may also want to enable regular archiving of your history file, to prevent the full history from being loaded into memory by each new bash shell. With a history file size of 10,000 entries, bash uses approximately 5.5MB of memory on Solaris 10, with no appreciable start-up delay (with $HOME on a local disk, I assume? -- GreyCat). With a history size of 100,000 entries this has grown to 10MB with a noticeable 3-5 second delay on startup. Periodic archiving is advisable to remove the oldest log lines and to avoid wasting resources, particular if RAM is at a premium. (My largest ~/.bash_history is at 7500 entries after 1.5 months.)

This is best done via a tool that can archive just part of the file. A simple script to do this would be:

•  #!/bin/bash
   umask 077
   max_lines=10000
  
   linecount=$(wc -l < ~/.bash_history)
  
   if (($linecount > $max_lines)); then
           prune_lines=$(($linecount - $max_lines))
           head -$prune_lines ~/.bash_history >> ~/.bash_history.archive \
                  && sed -e "1,${prune_lines}d"  ~/.bash_history > ~/.bash_history.tmp$$ \
                  && mv ~/.bash_history.tmp$$ ~/.bash_history
   fi

This script removes enough lines from the top of the history file to truncate its size to X lines, appending the rest to ~/.bash_history.archive. This mimics the pruning functionality of HISTFILESIZE, but archives the remainder rather than deleting it - ensuring you can always query your past history by grepping ~/.bash_history*.

Such a script could be run nightly or weekly from your personal crontab to enable periodic archiving. Note that the script does not handle multiple users and will archive the history of only the current user - extending it to run for all system users (as root) is left as an exercise for the reader.

Archiving by month

# https://github.com/kaihendry/dotfiles
mkdir -p ~/bash_history
shopt -s histappend
HISTCONTROL=ignoredups
PROMPT_COMMAND="history -a; history -n; $PROMPT_COMMAND"

# If your bash is older than 4.3, set these to a large number instead
# else your history files will be empty
HISTFILESIZE=-1 HISTSIZE=-1

HISTFILE=~/bash_history/$(date +%Y-%m)

h() {
    grep "$@" ~/bash_history/*
}

https://youtu.be/DJ_HdmfA72E

• What happens when the date changes to a new month while your shell is still running? Then HISTFILE is pointing to the wrong place.

Saving history into a database

TODO...

---