This is an archive of various mostly trivial programs, gathered here for myself and in case they prove useful to someone else.
check_app_versions.py is a Nagios plugin to report on app stack versions for Debian’s apt/dpkg, Red Hat’s yum/rpm, Node.js, PHP, Interchange, or from a file output by some other tool. It uses the old Python 2 to run on the system Python of CentOS 7.
ptr-check checks whether each of a list of DNS hostnames resolves to an IP address that has a PTR record that matches the specified hostname. It uses the dog DNS client to produce JSON output it parses.
pull-postgres2.sh is a bash program to SSH to a remote server, find the most recent PostgreSQL version of pg_dumpall, export the database, and tar over ssh some data to the system running it. I keep it for reference to remind myself how to do nested bash heredocs to pipe in a script and avoid quoting problems in ssh, which was tricky to get right.
postgres-database-size.txt is a SQL query and script to run it to report on the disk space used for a PostgreSQL table, indexes, TOAST data, and total.
geoip-lookup looks in the MaxMind GeoIP local database to find an IP address or hostname’s geographic location. Trivial, but a command like this oddly isn’t included.
extract-har-hostnames.bash extracts hostnames from URLs in HAR files exported by e.g. Firefox & Chrome.
binary-to-text.py and binary-to-text.pl convert textual binary to ASCII/Unicode equivalents.
inventory-json-objects-walktree.py and inventory-json-objects-filenames.py inventory datatypes of JSON files in a directory tree. They do the same thing; the first is object-oriented and walks directory trees itself, and the second is functional and relies on xargs to supply names (and exclude circular symlinks).
check-cloudflare-ips fetches Cloudflare’s list of source IP addresses, normalizes them, and saves, for comparison with the previously fetched version in a Git repo. This allows for being alerted when Cloudflare adds or removes source IP addresses for its proxy CDN service, so you can have firewalls deny access from any other source.
check_sitemap.py is a Nagios plugin for fetching an Google XML sitemap and checking it is recent enough and has a configurable number and content of links.
bind2tfcf.pl converts DNS info from BIND zone file format to Terraform Cloudflare provider configuration.
check_postfix_blocked.py is a Nagios plugin that scans Postfix logs (in syslog format) to count outbound messages blocked or greylisted as spam and alert based on supplied thresholds.
fix_latin_jon.pl converts a text format PostgreSQL pg_dump file from “byte soup” (mixed character set encodings) to UTF-8, leaving UTF-8 as is, converting other lines from Windows-1252 where possible. See also Postgres migrating SQL_ASCII to UTF-8 with fix_latin by Greg Sabino Mullane.
burncpu.pl and burnmem.pl are trivial, for using 1 CPU and growing resident memory used, respectively.
collect-server-stats is a simple shell script for collecting server statistics for diagnostics. The script gather-forensics-debian.sh is similar but designed to gather forensics into a tarball for easy downloading as part of a batch.
imagemunge and timestamp-rename are scripts to do lossless rotation of JPEG (with jpegtran) and rename JPEG, HEIC, and PNG files (using jpegtran or exiftool) to match either their last-modified, JFIF, or EXIF timestamps.
kill-large-processes: kills large processes (1 GiB by default).
idlekill: selectively kills processes of idle login sessions, simplifying unattended calls to skill.
pgindexsize: rebuild PostgreSQL indexes and report on disk space saved (or rarely, gained).
odbc_dump_dbi.pl and odbc_dump_win32.pl: dump ODBC-accessible database tables to tab-delimited text files using Perl on Windows. Microsoft’s Access table exporter (and by hearsay, SQL Server’s too) don’t filter tabs from the data, thus corrupting any row with a tab in one of its text fields. This program lets you dump everything cleanly. It converts tabs to spaces and makes sure no Unix line endings are in the data.
te: table editor, a front-end for editing tab-delimited text files with your favorite text editor. Now included as part of Interchange.
qx2html.pl converts QuarkXPress Tags text exports to HTML. Written for MacPerl, but easy to adapt.
Return to Jon's website home.