Solaris SVM metasets Comments Off on Solaris SVM metasets

The Solaris disk management/virtualisation tools have gone through many name changes – ODS, SDS, SVM – but the basic tools have remained the same. With the introduction of Sun Cluster, Sun needed to come up with a way to share storage between cluster nodes. Obviously this functionality needed to be added to SVM, and they came up with the idea of metasets.

Normally, metadevices are local to a host. When encapsulating disk slices into metadevices, you first have to dedicate a disk slice to store the metadatabase (with them metadb command). You can then create replicas of the metadb, and scatter them across slices, to ensure they don’t get corrupt and you lose all your metadevice information.

metasets are, in a nutshell, a collection of metadevices that have their own metadb state databases.

This works well in a cluster – the metaset metadevices, along with their metadbs, can be moved around, existing on the node which needs to mount their filesystems.

It also makes things easy for use when mounting LUNs from a SAN. If we can encapsulate the LUNs within metadevices, in their own metaset, then if our host dies we can just re-import everything on another host. Think of it as very basic – but very quick and easy – disaster recovery.

Think of metasets as being very similar to disk groups in Veritas Volume Manager.

Creating the metaset is a simple process. First of all we define our metaset and add our host to it.

bash-3.00# metaset -s test -a -h avalon

Syntax is pretty straightforward:

• -s is used to specify which metaset we’re using
• -a is the add flag. Guess what -d does?
• -h specifies the hostname which owns this metaset

All metasets are owned by at least one host (it’s how they track who can access them). If you’re in a cluster environment, multiple hosts will own the metaset, allowing the cluster software to move the metadevices between nodes.

For a single hosted metaset, however, we just need to add one host, and we need to make sure that it will automatically take ownership and import the metaset on boot.

All we have to do to make this happen is enable the autotake flag on the metaset:

bash-3.00# metaset -s test -A enable

And that completes the setup of the metaset. We then just select which LUNs we’re interested in, and add them in to the metaset:

bash-3.00# metaset -s test -a c7t60060E80141189000001118900001A10d0 \
c7t60060E80141189000001118900001A17d0 \
c7t60060E80141189000001118900001A17d0 \
c7t60060E80141189000001118900001A19d0

Note that when we add devices to a metaset (disks or LUNs) we need to only specify the device name – not slices, and not s2 (the Solaris way to reference an entire disk by a single reserved slice).

Normally, when you create a metadevice, you are encapsulation a slice that already exists on disk. This means the data stays intact. This is not the case when importing a disk into a metaset.

The act of importing a disk re-partitions it. All existing partitions are deleted, with a tiny slice on s7 being created to store the metadb replica, and the rest given over to s0. Note that s2 – the usual way of addressing a disk in Solaris – is also removed.

Here’s what the partitions look like on our root disk:

bash-3.00# prtvtoc /dev/dsk/c1t0d0s2
* /dev/dsk/c1t0d0s2 partition map
*
* Dimensions:
*     512 bytes/sector
*     107 sectors/track
*      27 tracks/cylinder
*    2889 sectors/cylinder
*   24622 cylinders
*   24620 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      2    00    8389656  23071554  31461209
       1      3    01          0   8389656   8389655
       2      5    00          0  71127180  71127179
       5      7    00   31461210  20974140  52435349
       6      0    00   52435350  18625383  71060732
       7      0    00   71060733     66447  71127179

And here’s what they look like on a LUN that’s part of the metaset:

bash-3.00# prtvtoc /dev/dsk/c7t60060E80141189000001118900001A10d0s0
* /dev/dsk/c7t60060E80141189000001118900001A10d0s0 partition map
*
* Dimensions:
*     512 bytes/sector
*     512 sectors/track
*      15 tracks/cylinder
*    7680 sectors/cylinder
*   13653 cylinders
*   13651 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      4    00      15360 104824320 104839679
       7      4    01          0     15360     15359

We can query the metaset and have a look at it’s contents, to check everything is OK:


bash-3.00# metaset -s test

/dev/dsk/c7t60060E80141189000001118900001A19d0 Yes


Once we’ve populated our metaset, we create metadevices as normal. The only extras we need when using the metainit command is that we need to specify which metaset we’re using, and that we’ll always be using s0.

Let’s create a single metadevice striped across all 4 LUNs in our metaset:

bash-3.00# metainit -s test d100 1 4 /dev/dsk/c7t60060E80141189000001118900001A10d0s0 \
/dev/dsk/c7t60060E80141189000001118900001A17d0s0 \
/dev/dsk/c7t60060E80141189000001118900001A18d0s0 \
/dev/dsk/c7t60060E80141189000001118900001A19d0s0

metainit works in the same way it’s always done – we need to specify the full path to the slice we’re using – but with the additional -s flag to tell metainit which metaset we want to add the metadevice to.

We can use the summary flag to metastat (sorry, Solaris 10 only) to show us the summary of what we’ve just configured:

bash-3.00# metastat -c -s test
dbt/d100     s  199GB /dev/dsk/c7t60060E80141189000001118900001A10d0s0 \
/dev/dsk/c7t60060E80141189000001118900001A17d0s0 \
/dev/dsk/c7t60060E80141189000001118900001A18d0s0 \
/dev/dsk/c7t60060E80141189000001118900001A19d0s0

metasets are an easy way to group together storage and filesystems in Solaris, especially where the storage is external to your host, and you’d like the flexibility of importing it to another host in the future – for example, as part of some DR work if the host fails.

Solaris 9 branded zones Comments Off on Solaris 9 branded zones

Solaris zones are great for handling all those dev and test environments that seem to breed like cockroaches. The problem recently has been that lots of these environments have been on old sun4u architecture machines (which were cheap), whereas they should be on shiny new sun4v architecture machines (which are great for this sort of work).

Enter branded zones. Unlike a normal zone, which installs from the global zone, a branded zone uses a flar (Solaris flash archive) to install a different Solaris OE. In this case I’m going to go through setting up a Solaris 9 sun4u branded zone on a Solaris 10 sun4v T2000.

First of all, we need to apply a quick kernel patch to Solaris 10. Currently I’ve got:

bash-3.00# cat /etc/release 
                       Solaris 10 8/07 s10s_u4wos_12b SPARC
           Copyright 2007 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                            Assembled 16 August 2007

For branded zones for this OE, we need to apply kernel patch 127111-01 at a minimum. Once that’s in place, it’s over to Sun’s website to download the branded zones support files.

There are two different packages – one to support Solaris 8 branded zones, and one for Solaris 9. Head on over to http://www.sun.com/software/solaris/containers/ and download the one you need.

Once you uncompress the TAR file, there’ll be a README, which explains minimum patch levels, and which version of the branded zones package you need to install. For Solaris 9 on the 8/07 release of Solaris 10, I need to install version 1.0 of the Solaris 9 containers.

Installation is pretty simple:

bash-3.00# cd /var/tmp/s9containers-bundle/1.0/Product/
bash-3.00# pkgadd -d `pwd` SUNWs9brandr SUNWs9brandu SUNWs9brandk

pkgadd will go away and do it’s thing – answer ‘yes’ to all the questions, and in a couple of minutes the packages are in place. And that’s it – you’re done! Time to install those branded zones.

Zone configuration is pretty standard, except we need to tell Solaris we’re installing a specific branded zone:

bash-3.00# zonecfg -z test-zone
test-zone: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:test-zone> create -b
zonecfg:test-zone> set brand=solaris9
zonecfg:test-zone> set autoboot=false
zonecfg:test-zone> set zonepath=/export/zones/test-zone
zonecfg:test-zone> add net
zonecfg:test-zone:net> set physical=e1000g1
zonecfg:test-zone:net> set address=192.168.13.251
zonecfg:test-zone:net> end
zonecfg:test-zone> verify
zonecfg:test-zone> info
zonename: test-zone
zonepath: /export/zones/test-zone
brand: solaris9
autoboot: false
bootargs: 
pool: 
limitpriv: 
scheduling-class: 
ip-type: shared
net:
 address: 192.168.13.251
 physical: e1000g1
zonecfg:test-zone> verify
zonecfg:test-zone> commit
zonecfg:test-zone> exit

Note that the only real difference during zone configuration is branding the zone. Normally we wouldn’t specify the brand, and it would default to ‘native’.

Once the zone is configured, we need to install it. As we’re installing a branded zone, we can’t take the Solaris files from the global zone, so we need to use a flar as our boot image.

The process is very straightforward – we just supply the path to the flar as an argument to our zone install command:

bash-3.00# zoneadm -z test-zone install -u -a /mnt/flars/sun4u_sol_9_Generic_118558-11.flar 
      Log File: /var/tmp/test-zone.install.4436.log
        Source: /mnt/flars/sun4u_sol_9_Generic_118558-11.flar
    Installing: This may take several minutes...
Postprocessing: This may take several minutes...

        Result: Installation completed successfully.
      Log File: /export/zones/test-zone/root/var/log/test-zone.install.4436.log

And we’re done. Now just boot the zone and carry out the initial configuration as per normal.

Finally, a quick check to show off our sun4v architecture machine running a Solaris 9 sun4u branded zone:

bash-3.00# uname -m
sun4v
bash-3.00# cat /etc/release 
                       Solaris 10 8/07 s10s_u4wos_12b SPARC
           Copyright 2007 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                            Assembled 16 August 2007
bash-3.00# zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   5 test-zone        running    /export/zones/test-zone        solaris9 shared
bash-3.00# uname -a
SunOS sunt2k-test 5.10 Generic_127111-01 sun4v sparc SUNW,Sun-Fire-T200

RBAC examples – rebooting a server Comments Off on RBAC examples – rebooting a server

So, in a followup to the Solaris RBAC configuration post, I wanted to show how quick and easy it is to configure RBAC. As an example, I’m going to be working with the Solaris reboot command, on the basis that many developers want to reboot their environments, but you don’t always want to give them root.

So, the basic steps are:

1. define a Profile
2. assign a command to the Profile
3. define a Role
4. assign the Profile to the Role
5. allow a user to use the Role

Easy stuff. First stage, let’s create the profile. Profiles live in /etc/security/prof_attr, and are a way to group together similar commands. If you look in that file, you’ll see a lot of existing profiles, which tie together common groups of Solaris commands.

Adding a new profile is easy – we just add an extra line to the end of that file:

# echo "Reboot:::Profile to reboot Solaris:help=" >> /etc/security/prof_attr

Breaking it down – the first field is the profile name, and the fourth field is the description. The rest of the fields don’t matter at this stage, for what we’re doing.

The new profile is useless without a command, so let’s add the Solaris reboot command. Commands associated with RBAC profiles live in /etc/security/exec_attr (can you see a pattern in the filenames yet?) and – again – this file is pre-populated with command Solaris commands, grouped by profile.

# echo "Reboot:suser:cmd:::/usr/sbin/reboot:euid=0" >> /etc/security/exec_attr

Breaking the fields down again:

• first field is the profile name
• second field is the security policy – in this case, standard superuser
• third field is the type – in this case, it’s a command
• sixth field is the full path to the command
• final field is the effective user ID the command is executed as

So far, it’s all pretty straightforward. Now we have a profile, and we have a command associated with that profile. Now we need to create a role.

RBAC roles are essentially normal user accounts, which have a restricted shell, and associated profile(s). The restricted shell is there to apply all the execution privilege and audit trail RBAC goodness.

Adding a role is nice and easy:

# roleadd -m -d /export/home/reboot reboot
64 blocks

Note the command line options to roleadd are the same as used when adding a normal Solaris user with useradd.

We also need to give the role a password:

# passwd reboot
New Password:
Re-enter new Password:
passwd: password successfully changed for reboot

And now we can see the role has been added to /etc/passwd:

# grep reboot /etc/passwd
reboot:x:1001:1::/export/home/reboot:/bin/pfsh

So it looks almost exactly the same as a normal Solaris user. Now all we need to do is add a profile to the role. We do this with the rolemod command, which – again – is very similar to the normal usermod command:

# rolemod -P Reboot reboot

Details of which profiles are assigned to roles – and which roles are assigned to users – live in /etc/user_attr – so we can look in there to see the changes we’ve made:

# grep reboot /etc/user_attr
reboot::::type=role;profiles=Reboot

Finally we’ll add the role to our user account:

# usermod -R reboot tomk
UX: usermod: tomk is currently logged in, some changes may not take effect until next login.

And just look in /etc/user_attr to make sure the changes have been made:

# grep reboot /etc/user_attr
reboot::::type=role;profiles=Reboot
tomk::::type=normal;roles=reboot

We can use the roles command to see what roles we have available to us:

$ roles
reboot

However, logged in as myself I still can’t reboot the machine:

$ /usr/sbin/reboot
reboot: permission denied

And that’s because the profile is assigned to the role, not to my user account:

$ profiles
All
Basic Solaris User

The clue on how to use roles was in how they are created and stored – they’re just like normal users. So to access a role, we su to it:

$ su reboot
Password: 

The moment we su to a role, the whole RBAC audit trail kicks in. Everything, from that initial su onwards, is logged and tracked. Unlike sudo, this logging continues, even if we change shells or become another user (if the role allows us to). It’s this unbreakable audit trail that makes RBAC so powerful.

Now that we’ve assumed a role, we can check out the profiles available to us:

$ profiles
Reboot

So we can now execute the reboot command and bounce the box:

$ /usr/sbin/reboot
Connection to 192.168.13.101 closed by remote host.
Connection to 192.168.13.101 closed.

Have a look at the configuration files and see all of the roles and profiles that come pre-configured with Solaris. Play about with them and get familiar with the terminology. RBAC isn’t difficult or complex – it’s just very different. Get comfortable with it and you’ll soon be able to leverage it’s power to really secure your Solaris machines without denying users any functionality.

What version of the SAN Foundation Suite is installed? 1 comment

This is a constant pain that rears it’s ugly head again and again. You have a Solaris machine, the SAN Foundation Suite is installed, and you want to find out what version it is.

Well, you’d do a pkginfo on the SUNWsan package, right? Wrong.

# pkginfo -l SUNWsan
   PKGINST:  SUNWsan
      NAME:  SAN Foundation Kit
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  1.0
   BASEDIR:  /
    VENDOR:  Sun Microsystems, Inc.
      DESC:  This package provides a support for the SAN Foundation Kit.
    PSTAMP:  sanserve-a20031029172438
  INSTDATE:  Nov 24 2008 17:46
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed

Every version of SUNWsan reports 1.0. This is unforgivably rubbish – why hasn’t it been sorted yet?

To find out the real version of the SFS, you need to do something far more esoteric – look at the versions of the relevant kernel modules.

If you’ve got SFS 4.3 or later, they will be reported as ‘build dates’:

# modinfo | egrep '(SunFC|mpxio|scsi_vhci)'   
  79  138f975  15f10 149   1  fp (SunFC Port v20051027-1.61)
  80  13a50cd   7fa4   -   1  fctl (SunFC Transport v20051027-1.33)
  82  13ab699  5032a 153   1  qlc (SunFC Qlogic FCA v20050926-1.50)
  113 78196000  20313 150   1  fcp (SunFC FCP v20051027-1.80)
  114 781b8000   55fc   -   1  mpxio (MDI Library v20051027-1.17)
  115 781be000   c8cc 189   1  scsi_vhci (SCSI vHCI Driver v20051027-1.40)
  222 780de000   783f 154   1  fcip (SunFC FCIP v20050824-1.28)

If the version of SFS is earlier than 4.3, then the version is directly referenced for each driver:

# modinfo | egrep '(SunFC|mpxio|scsi_vhci)'   
  85 103199bc  10c23 149   1  fp (SunFC Port v5.e-2-1.18)
  86 1032a0c7   6f28   -   1  fctl (SunFC Transport v5.e-2-1.16)
  87 1032f747  2db28 153   1  qlc (SunFC Qlogic FCA v5.e-2-1.16)
  88 7825a000   fe94 150   1  fcp (SunFC FCP v5.e-2-1.17)
  89 7826a000   49ac   -   1  mpxio (MDI Library v5.e-1-1.7)

If you want to decipher what versions of SFS the build dates map to, you need to look at Sunsolve Document ID 216809. It can be found at http://sunsolve.sun.com/search/document.do?assetkey=1-61-216809-1

You seem to need a Sunsolve login (free) to view the document, and as far as I can tell it’s not restricted to contract customers only. If you find out that’s not the case, please let me know, and I’ll stick a copy up here.

Solaris RBAC configuration 1 comment

RBAC is Roles Based Access Control for Solaris. It’s similar to sudo in that it allows you to let specific users enhance their privileges to run specific tasks.

RBAC has been around in Solaris for a long, long time. It’s matured nicely and is now very accessible and usable – once you get your head round the concepts. It’s very powerful and it’s an under-used framework which really needs to be understood more.

Unlike sudo RBAC can be integrated into NIS or LDAP, and provides a much more comprehensive framework out of the box for doing things.

The basic tasks to setup RBAC are:

1. define a ‘Profile’ in prof_attr
2. tie specific commands to that profile in exec_attr
3. tie specific authorisations to that profile in auth_attr
4. tie the profile to a role, then that role to a user(s) in user_attr
5. or tie the profile directly to a user in user_attr

There are some tradeoffs between using Roles vs. adding Profiles to user:

• Roles require a user to su to another user to execute the command
• when many users need to do the same thing that requires several different Profiles, it’s easier to change the Role once to add more Profiles than to edit many users to add the same extra Profiles to all of them
• easier to track who has access to what by using Roles rather than assigning things direct to users

Personally, when doing more than one thing, I like to define a Role, and then make users su to that Role. It makes management and scalability that much easier, and also more clearly defines who can do what.

It also continues the paradigm of having to su to another user to gain extra privileges. Keeping things working the same as they have done previously is the easiest way to introduce new methods of working, and new technologies, to the users whilst avoiding complaints :-)

In the next few posts I’ll cover some basic RBAC setup, and then adding RBAC configuration to allow a non-privileged user to control Solaris services via SMF.