pipelog - pipe input to a file which can be rotated via signals
pipelog OUTPUT_FILE_PATH [ options ]
pipelog [-h | -v]
Pipelog will read text data from standard input and write to OUTPUT_FILE_PATH. On receipt of SIGUSR1, it will rotate OUTPUT_FILE_PATH to OUTPUT_FILE_PATH.1, and move any existing .1 file to .2 and so on. For example:
$ myapp | pipelog myapp.log &
$ kill -s USR1 `pidof pipelog`This assumes you only have one pipelog running; if not, you can probably tell them apart with ps -o pid,args -C pipelog, but see also the -l and -p options below. OUTPUT_FILE_PATH should not begin with a '-', or it will be mistaken for an option.
On receipt of SIGUSR2, pipelog will delete the current log file and open a new one with the same name, effectively clearing the contents. In both cases (rotation or deletion), pipelog will not break a line of input; i.e., if there is input buffered without a newline, that will not be written out until either a newline is received or the input stream terminates.
Unless you use the -a switch, if OUTPUT_FILE_PATH exists when pipelog starts it will be rotated immediately.
Pipelog logs some messages of its own, separately (see MESSAGES below). Note that since pipelog uses C string functions internally, non-text data which contains zero bytes will have negative consequences on its functioning; see the -z option if you think this might be a problem.
Pipelog always flushes up to the last newline so the content of the log should be observable in real time. If you suspect a delay in logging, it is most likely because you are piping the standard output of a process that will flush on newline when connected to a console, but otherwise buffers a small amount. I.e., the delay is in pipelog receiving input; set your application to not buffer standard out.
Pipelog currently does not support mashing switches together, so you must use, e.g. -a -f, and not -af.
Initially, pipelog will open the log file and append to it, if it exists.
Print messages (see MESSAGES below) to standard error instead of syslog.
Print a usage message to standard output and exit.
'mode' should be an octal permissions mode, e.g., 600. This will be used to set permissions on all new log files (and any existing one, if combined with -a). Since pipelog itself creates these files, if the permissions do not include owner-write, this will fail with a usage message.
Whenever a new log file is created, pipelog will prepend a line including its executable name (presumably, 'pipelog'), its PID, and the current date and time.
At start-up, pipelog will write its PID to 'filepath'. No newline is added, so you can put this into a shell variable with, e.g., PIPELOG_PID=`cat filepath` or, using bash, PIPELOG_PID=$(<filepath). On a normal exit (i.e., when the logged process dies and input stops), pipelog will remove this file.
Normally, pipelog writes a brief message to standard output when it starts up. This will suppress that and two of the messages under MESSAGES below, 'Rotated output file' and 'Received EOF from input, exiting'.
Print the version number and exit.
When a log file is rotated (excepting at start-up if the log pre-exists), pipelog will execute "command" after the new log is created (and a line prepended, if -l was used) and the old log rotated. It will continue to buffer input but will not write anything out until this command exits. This allows for, e.g., prepending a header to the log file, or tailing part of 'mylog.1' to the beginning of my 'mylog' to create some overlapping context. The standard input, standard output, and standard error stream of this process will be closed from the start, so do not try and include redirection in the command. If you need to use these streams, wrap your stuff in a shell script and use that as 'command'. You can quote the command to include whitespace, and use escaped quotes, e.g., -x "foo -y \"hello world\"".
Scan input for zero bytes and replace them with 'character', which must be from the ASCII charset (or otherwise one byte in length, if your locale permits this). Text should never contain null (zero) bytes, and so for normal logging you should not need to do this.
Messages from pipelog itself are kept separate from whatever input is being logged to a file; by default these go to syslog using facility USER and a priority ranging from NOTICE to CRITICAL (the only critical error is "Could not allocate memory!"). They can be sent to standard error instead with the -f switch. In either case, messages are always preceeded with the name of the executable (presumably, 'pipelog') and the PID. In some cases there may be a further detail, after the message, as reported by strerror(errno).
Messages labelled 'FATAL' indicate pipelog exited subsequently, these are listed last below.
fclose() on the log handle failed prior to rotation or deletion.
This happens if you use the -p switch and pipelog could not create the pidfile.
This is noted on a normal exit if there is buffered data that could not be written out. This should not usually happen.
The fork() system call prior to executing a -x command failed -- theoretically possibility.
Pipelog will fail on 3 consecutive fwrite() errors to the log file; this is the second one.
This not an error, but an indication that the input stream terminated and pipelog is done.
This is not an error, but an indication that the log was rotated on SIGUSR1 or cleared on SIGUSR2.
A rotate or delete signal was received before the last -x command exited, and will not be acted upon.
This is very unlikely, but perhaps not impossible if you manage to send two signals nearly simultaneously.
Pipelog will fail on 3 consecutive fwrite() errors to the log file; this is the first one.
This indicates 3 consecutive fwrite() errors on the log file handle.
A request for additional memory (e.g. via malloc()) failed. Unless the process providing input produces absurdly long lines, pipelog is unlikely to use a significant amount of memory.
unlink() on the log file failed after reciept of SIGUSR2.
Possibly someone changed the write permissions on the directory where the logs are being rotated since pipelog started, or deleted it completely.
fopen() on a new log file failed.
Reading from the input stream failed.
This might happen if someone has changed the read permissions on the directory where the logs are being rotated since pipelog started.
Normally, pipelog exits with a status of 0. Other possible values are:
For all of them except 1, there should be an error message logged with further details (see MESSAGES above).
http://cognitivedissonance.ca/cogware/pipelog
Copyright 2014 Mark T. Eriksen