UNIX Consulting and Expertise
Golden Apple Enterprises Ltd. » Posts in 'Scripts' category

OpenSSL tricks – checking https ports Comments Off on OpenSSL tricks – checking https ports

Checking whether or not your web server is running is pretty simple – telnet to port 80, issue a HEAD request, and make sure you get a valid response. What’s less well known is how to test an https session – in this post I’ll go through the nice tool the OpenSSL toolkit gives us.

People think of OpenSSL as a collection of libraries that enable us to build in SSL support to a variety of things – webservers, LDAP servers, etc. OpenSSL also happens to be a toolkit in binary form that’s built along with the libraries, and it’s a pretty powerful bit of kit.

First of all, we can use the s_client functionality to test an https connection:

bash-3.2$ openssl s_client -connect www.siliconbunny.com:443

CONNECTED(00000003)
depth=0 /C=GB/ST=Berkshire/L=Crowthorne/O=Silicon Bunny/CN=www.siliconbunny.com/[email protected]
verify error:num=18:self signed certificate
verify return:1
depth=0 /C=GB/ST=Berkshire/L=Crowthorne/O=Silicon Bunny/CN=www.siliconbunny.com/[email protected]
verify return:1
---
Certificate chain
 0 s:/C=GB/ST=Berkshire/L=Crowthorne/O=Silicon Bunny/CN=www.siliconbunny.com/[email protected]
   i:/C=GB/ST=Berkshire/L=Crowthorne/O=Silicon Bunny/CN=www.siliconbunny.com/[email protected]
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDpjCCAo4CAQAwDQYJKoZIhvcNAQEEBQAwgZgxCzAJBgNVBAYTAkdCMRIwEAYD
VQQIEwlCZXJrc2hpcmUxEzARBgNVBAcTCkNyb3d0aG9ybmUxFjAUBgNVBAoTDVNp
bGljb24gQnVubnkxHTAbBgNVBAMTFHd3dy5zaWxpY29uYnVubnkuY29tMSkwJwYJ
KoZIhvcNAQkBFhp3ZWJtYXN0ZXJAc2lsaWNvbmJ1bm55LmNvbTAeFw0wNDA4MTcy
MjExMTJaFw0xNDA4MTUyMjExMTJaMIGYMQswCQYDVQQGEwJHQjESMBAGA1UECBMJ
QmVya3NoaXJlMRMwEQYDVQQHEwpDcm93dGhvcm5lMRYwFAYDVQQKEw1TaWxpY29u
IEJ1bm55MR0wGwYDVQQDExR3d3cuc2lsaWNvbmJ1bm55LmNvbTEpMCcGCSqGSIb3
DQEJARYad2VibWFzdGVyQHNpbGljb25idW5ueS5jb20wggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQDfMrAWj1SU9IKclxLucFaptJZ3eehlV7VI+gNjeIV1
bLTaBvIR/k4e2dYArG05ZZwXu+N8zvqDQfzJaFV4z5a/1nGjzx63VexRM6ix79NX
I1/hJ3m6qU+C0Iy0lbKIy60r8F3DS5N+URZDHyQTb0zV9c9+4WFPpc6J+zDeRP0r
WfF2CDzSV16snTonLzSe7NAL0Br9+5sp26vERX0+Syg1pSG15c3YKMnaNbF8dquE
UC1SAso8PPgKdw9Fle38ulWbX6Lr6gXEWvo1Vb61SEiTz+pcv4fPXtHix4knifIF
kqs+rvELYCGuIWV0gTFuF3/opGazSRub8U2Af9QjjJvdAgMBAAEwDQYJKoZIhvcN
AQEEBQADggEBANnYch3XCiteZRnzeVtmmmDjpn7OVfc/V9QzL8mVTVdopd/oVScI
4gyz2lDpLxiLeXA86WDpai2rys7aXGjwIBtSaHHt1O5bYG6kR+H3RLunklIaJhBr
0bj9Hffa15POEEctZdfMO3OJ/4nR8Bd6lJRllvF4qkDsbKnwEXd/IB33sVtvtNxM
LUAgtq8BaeLysUMxVRbsFJ01J6uyv6Y6OtzY3QedkP+ig3+IkWqgPm1zTh1CflSH
OUXeMQnPl8mN6s7deg1WBmiSgc6um3Tb5NoL1CmRPamJip5DqPffR+EboxReRiRJ
fofgD0cIKKghQsHXxWL8OcqrfDQpC9BgUWw=
-----END CERTIFICATE-----
subject=/C=GB/ST=Berkshire/L=Crowthorne/O=Silicon Bunny/CN=www.siliconbunny.com/[email protected]
issuer=/C=GB/ST=Berkshire/L=Crowthorne/O=Silicon Bunny/CN=www.siliconbunny.com/[email protected]
---
No client certificate CA names sent
---
SSL handshake has read 1630 bytes and written 316 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 80F355438981C329BEF0BB1CCA4936906EE0A0F71C0B7AD4A873629081E7452A
    Session-ID-ctx: 
    Master-Key: BD04DC16B134FBB2B5F5833FEB72853245EC060536AD6F4A6FEBA7DFD47F607693795F9CE3B1F291593E489B685FAE70
    Key-Arg   : None
    Start Time: 1260328910
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---

This allows us to form a proper SSL connection to the web server – we can see the certificate, check it’s validity, and then run our HEAD request check as well. We’re not just doing a basic “are you listening?” check – openssl is forming the same https connection a client would, so this is very handy when checking out certificate mis-matches or bizarre client errors.

Within the same session we can then start talking http and check our server is doing the right thing:

HEAD / HTTP/1.0

HTTP/1.1 200 OK
Date: Wed, 09 Dec 2009 02:02:43 GMT
Server: Apache
X-Pingback: http://grond.gaeltd.com/xmlrpc.php
Cache-Control: max-age=0
Expires: Wed, 09 Dec 2009 02:02:43 GMT
Vary: Accept-Encoding,User-Agent
Connection: close
Content-Type: text/html; charset=UTF-8

closed

Worst case you can do this testing direct on your web server, but pretty much most machines should have OpenSSL installed, and at a minimum you should look at adding it to your collection of tools on your laptop or memory stick.

Another interesting and lesser known use of OpenSSL is for file encryption.

This is an example using OpenSSL’s enc function to encrypt a text file using the Blowfish cipher:

bash-3.2$ openssl enc -e -a -salt -bf -in testfile.txt -out testfile.blowfish
enter bf-cbc encryption password:
Verifying password - enter bf-cbc encryption password:

You’re prompted twice to enter a password to be used, then OpenSSL will encrypt the file for you.

Decrypting a file is very similar – calling the enc function in decrypt mode (-d) and changing your input and output files:

bash-3.2$ openssl enc -d -a -bf -in testfile.blowfish -out tomcat-testfile.txt
enter bf-cbc decryption password:

Using OpenSSL like this for file encryption gives you simple, easy access to quite strong encryption algorithms, but without the hassle of managing key files that you get with PGP – so can be an ideal solution for things like managing sensitive webserver log files.

The last OpenSSL trick to look at is hashing functions – specifically we want to calculate a message digest to check that a file hasn’t been tampered with.

Although outdated md5 is still the most commonly used hash function to check a file’s integrity – most often you’ll be looking at md5 checksums to verify a large file has been fully downloaded, or that it’s not been tampered with.

All we need to do is call OpenSSL with it’s digest function, specify the hash algorithm to use, and then give it a file to check. Classic case here – I want to verify that the checksum for the VPN software I’ve downloaded matches up:

bash-3.2$ openssl dgst -md5 -c Tunnelblick_3.0b22.dmg 
MD5(Tunnelblick_3.0b22.dmg)= 5b:d3:6d:2a:06:22:9f:58:00:01:f8:e1:15:48:7c:d9

Although md5 is the most common hash function in use, it’s considered outdated and has been deprecated in favour of stronger functions like SHA-1 – which are just as easy to use via OpenSSL:

bash-3.2$ openssl dgst -sha1 -c Tunnelblick_3.0b22.dmg 
SHA1(Tunnelblick_3.0b22.dmg)= 7f:56:1c:96:68:4a:fc:b3:f6:27:99:11:41:89:ed:7e:30:97:28:7f

Hopefully this has given you an idea of the power and flexibility of the OpenSSL tookit. A big advantage of utilising OpenSSL in this way is that it can easily be scripted, given you some very powerful tools for carrying out simple sanity checks on remote, publicly accessible servers.

Getting your scripts to log to syslog Comments Off on Getting your scripts to log to syslog

A constant problem when people write scripts is that you end up with loads of different log files scattered across the file system. This brings with it the associated pain of parsing the log files, archiving the old ones, etc. etc.

Wouldn’t it be great if you could get your scripts to log to syslog? Enter logger, which is present on pretty much all UNIX systems.

At the top of your script, after defining the Korn shell (you are writing in Korn, aren’t you? You do expect your scripts to work across more than one platform, don’t you?) you can add the simple construct:


logger -p daemon.notice -t ${0##*/}[$$] |&

exec >&p 2>&1

And behold! Magical script entries in syslog – in this example, from a script called test_script running on an Origin 200 called frith:

Nov 25 17:40:41 frith test_script[17449]: [ID 702911 daemon.notice] scripty logging goodness

The IRIX manpage for logger says:

Logger provides a shell command interface to the syslog(3B) system log
routine. It can log a message specified on the command line, from a
specified file, or from the standard input. Each line in the specified
file or standard input is logged separately.

The Solaris manpage is a bit more verbose:

The logger command provides a method for adding one-line
entries to the system log file from the command line. One or
more message arguments can be given on the command line, in
which case each is logged immediately. If this is unspeci-
fied, either the file indicated with -f or the standard
input is added to the log. Otherwise, a file can be specified, in which case each line in the file is logged. If neither is specified, logger reads and logs messages on a
line-by-line basis from the standard input.

However the important thing is that logger takes the same key options and works in the same way – giving you a simple, portable way to get syslog entries from your custom scripts, cross platform.

Scripts and Tools Comments Off on Scripts and Tools

This is a collection of scripts and tools developed by us, which are freely available for download.

All scripts and tools are copyright © G.A.E. Ltd
You may distribute under the terms of the GNU General Public License with the exception that it cannot be placed on a CD-ROM or similar media for commercial distribution without the prior approval of the author.
This code is provided with no warranty of any kind, and is used entirely at your own risk.

More tools, FAQs, and information can be found on SiliconBunny, our Silicon Graphics information site.

ods_breakage.ksh

Description: Used on hosts with Solaris Disksuite volume management. The script parses metastat output and checks the status of metadevices. Any errors are collated and a status report is emailed.
Platform: Solaris
Installation: Edit the SYSADMINS variable, changing it to an email address where you want alerts sent.
Usage: Run from cron every hour, takes no command line options
Download: ods_breakage.ksh

ce_settings.ksh

Description: Sun’s Cassini Ethernet (ce) cards are available in either 100mb or 1gb versions. The same driver is used for both. Problems arise because speed and duplex settings cannot be set from within /etc/system. Instead, a configuration file must be used. The syntax is cryptic and prone to mistakes – at the worst case, a poorly written configuration file can cause a kernel panic on bootup. This script will parse /etc/path_to_inst and generate a ce.conf configuration file, forcing 100mb ce cards to 100mb/full duplex, and letting 1gb cards auto-negotiate.
Platform: Solaris
Installation: Copy the script somewhere. Edit the PATH_TO_INST and OUTPUT variables to point to /etc/path_to_inst and where you want your ce.conf to be placed (usually somewhere like /kernel/drv/ce.conf)
Usage: Once the above variables have been set, run the script. A new ce.conf file will be generated, and the settings will take effect the next time the machine is booted.
Download: ce_settings.ksh

mem_dis.ksh

Description: During performance testing on F15k domains, it was required to remove memory from several system boards to gauge relative performance when increasing CPU count, memory, and both together. Rather than physically removing the boards, this script was developed. It uses the Sunfire’s DR to unconfigured memory from selected system boards.
The operation of the script is very simple, and can be used as the basis to automate many DR commands on Sunfire machines.
Platform: Solaris
Installation: Place the script somewhere in a Sunfire domain. Must have root privileges to run.
Usage: Run the script. It will display a listing of memory on system boards, and ask you what you want to disable. Enter the details, and the script calls cfgadm to disable that memory.
Download: mem_dis.ksh

if_check.sh

Description: Before IP MultiPathing (IPMP) came along in Solaris 8, it was difficult to have 2 ethernet interfaces connecting your host to the same LAN in a failover configuration. This script pings a highly-available source (usually your default router or switch) and calls ifconfig to down an interface and up the spare if the ping fails. Your two interfaces should be connected to seperate switches for maximum availability.
Platform: Solaris, ideally pre-8
Installation: Copy the script somewhere, and run from root’s crontab every 5 or 10 minutes, depending on how much of a delay you can handle when an interface fails.
Usage: The script takes one argument, the IP address to ping.
Download: if_check.sh

clone_root_disk.ksh

Description: While IRIX comes with the XLV volume manager, the license to mirror (plex) your volumes is an expensive extra. This script allows you to clone the root disk, creating a bootable spare which is not normally mounted, allowing easy root disk recovery.
Platform: IRIX
Installation: Copy the script somewhere and run from root’s crontab. You may need to edit the disk device names to match your machine’s configuration.
Usage: Run from root’s crontab. The script takes no arguments.
Download: clone_root_disk.ksh

int_check.ksh

Description: Tired of remembering and typing in the ndd variables to check the speed and duplex settings of an ethernet port? This script automates the drudge work – just give it an interface type (hme, ce, bge) and an instance number, and it will tell you how that port is configured.
Platform: Solaris
Installation: Place the script on your host and run as root.
Usage: Run as root. It will take two command line arguments – the interface type, and the instance number.
Download: int_check.ksh

kstat_check.ksh

Description: kstat can return a lot of useful information about the state of your ethernet interfaces. However, the syntax can be confusing and difficult to remember. This script functions as a wrapper around kstat – tell it the interface type and instance number, and it will return configuration details. It’s important to note that older cards, like hme, will return less information than newer cards, like ce. This is due to driver improvements as opposed to deficiencies in kstat.
Platform: Solaris
Installation: Place the script on your host and run as root.
Usage: Run as root. It will take two command line arguments – the interface type, and the instance number.
Download: kstat_check.ksh

split.ksh

Description: This script is an example of what can be done with EMC’s Symcli tools and Symmetrix Business Continuance Volumes (BCVs). In this example, an Oracle database is running on the main Symmetrix disks. The BCVs are mounted on another host, which contains a datawarehouse. The script unmounts the BCV volumes, mirrors them to the main disks, and then splits them off again, remounting them on the host. You then have a hot copy of your live Oracle database which can be imported into your datawarehouse without any impact on the production system.
Note that there are many variables involved (volume types, hosts, Oracle database, etc.) and so this script should be taken as an example of what it is possible to do.
Platform: Solaris host with Veritas Volume Manager, connected to an EMC Symmetrix, with Symcli installed
Installaton and Usage: Don’t! Read the script, understand what it does, and then use that as the basis for your own scripts, customised for your own needs.
Download: split.ksh

Top of page / Subscribe to new Entries (RSS)