Jay Taylor's notes

back to listing index

Rotating logs generated by a process that logs to stdin

[web search]
Original source (serverfault.com)
Tags: logrotate daemontools djb serverfault.com
Clipped on: 2018-04-09

I have a long running process, that writes its log file to stdout. I would like to save this output to different files, automatically maintain these files (like deleting/archiving the old ones), without restarting the main process.

On solution would be to send the output to a file (process > log.txt), and use logrotate on it, but logrotate needs to restart the program, which is not possible.

The other one is to pipe the output to cronolog (process | cronolog), but in this case older files won't get deleted / archived, meaning I have to make a program that will do the maintenance for me.

The best would be to be able to use both utilities, because with cronolog I don't need to restart the process, and logrotate will maintain the old log files exactly as I want. Is there a way to get these two programs to work with each other? If not, what is a good solution to this problem?

asked Nov 3 '12 at 18:58
SztupY
15512
up vote 3 down vote accepted

Multilog from DJB's daemontools can do (almost) exactly what you are asking. The only downside that I am aware of is that not many distributions carry a package for daemontools.

If you are not managing your application with svc (part of the daemontools) you will have to find a way to pipe the output to a command like

multilog t s1048576 n100 ./my_log_directory

This translates to:

  • t: insert a tai64n timestamp (that can be translated to readable time with tai64nlocal)
  • s1048576: rotate the log file when it grows to 1MiB
  • n100: keep no more than 100 rotated files
  • ./my_log_directory: anything starting with . or / - write the log to that directory

The log that is written will have the filename current, and when multilog rotates the log, it will be renamed to @<tai64n timestamp>.s where the filename shows the time when the file was rotated. The extension can be .s if the file was safely flushed, or .u if it may have been truncated.

For more info, just check the links.

answered Nov 5 '12 at 9:01
chutz
5,6061643

Your Answer

 
community wiki

Not the answer you're looking for? Browse other questions tagged or ask your own question.

asked

5 years, 5 months ago

viewed

2,259 times

active

5 years, 5 months ago

Hot Network Questions

more hot questions