Add EL7 support for xrootd-hdfs

Description

From SOFTWARE-2162, xrootd-hdfs-1.8.7-1.osg33.el7 is not quite ready for prime-time, and xrootd-4.2.3-1.osg33.el7 will most likely also need to be fixed for el7 systemd support.

Freshdesk Tickets

None

Activity

Show:
Carl Edquist
February 11, 2016, 7:31 PM

Linking to documentation ticket, SOFTWARE-2206.

Tim Cartwright
February 11, 2016, 6:03 PM
Edited

Yes, cleared to promote both xrootd-hdfs-1.8.7-1.osg33.el*. (Sorry for the original version of this comment, where I incorrectly cleared OSG 3.2 promotions, too.)

Please make a separate ticket, labeled for March, to provide an initial installation guide for this software. Work with and me to get started, but expect to do the bulk of a first draft, plus revisions, yourself. The documentation is higher priority than updating Hadoop itself.

Carl Edquist
February 11, 2016, 1:11 AM

Promotion request for .

Well, my comment above shows how little I know about systemd \:)

Turns out the "@" at the end of the service name means it's an "instanced" service, with multiple instances possible. Trying to start a service named xrootd or xrootd@ does not work, but xrootd@_instancename_ does. (Thanks for the help with that!)

Systemd will allow any instance name, but the xrootd server will expect to find a corresponding /etc/xrootd/xrootd-instancename.cfg, or else it will fail to start up. The /etc/xrootd directory comes with an xrootd-clustered.cfg and an xrootd-standalone.cfg, so two options are 1) to copy the xrootd.cfg with the hdfs settings onto xrootd-clustered.cfg and start the xrootd@clustered service, or 2) to just make a new xrootd-hdfs.cfg with the hdfs settings and start a new xrootd@hdfs service instance. I ended up doing the latter.

For verification, first I created a test file on a separate node via a hadoop fuse mount, then on the xrootd node
I was able to run "xrdcp root://$HOSTNAME//test-file test.out" and confirmed the contents of the transferred test file was as expected.

So, xrootd-hdfs is working for el7, but the configuration/startup procedure is a little different with systemd, so we should probably document that somewhere. Actually, I don't think we have much of our own documentation on xrootd-hdfs at all (even for pre-el7/systemd). I found some bits and pieces of reference material in the following docs:

https://twiki.opensciencegrid.org/bin/view/Documentation/Release3/InstallHadoop200SE
https://twiki.opensciencegrid.org/bin/view/Documentation/Release3/InstallXrootd
https://twiki.cern.ch/twiki/bin/view/Main/HdfsXrootdInstall

For el6, I did the standard 3 developer test installs of xrootd-hdfs-1.8.7-1.osg33.el6 – I just verified the installs/upgrade installs, as the effective changes since the current production release are el7-specific. There's also a set of .osg32 builds, but I presume we will not plan to ship them now, as, again, the changes are el7-specific.

, is it OK to promote xrootd-hdfs-1.8.7-1.osg33.el* ?

Carl Edquist
February 10, 2016, 7:38 PM

Pasting my xrootd comment from SOFTWARE-2162:

The xrootd-server package (not xrootd-hdfs) doesn't seem to be set up properly for systemd – there is a "/usr/lib/systemd/system/xrootd@.service" (I'm guessing the "@" is a mistake), which prevents starting the service normally. (I tried making a symlink without the "@", but that failed also.) Again, this isn't necessarily a problem with xrootd-hdfs, but (as far as my notes say) it's required for xrootd-hdfs testing, and in any case it probably needs to be fixed in the xrootd packaging.

Fixed

Assignee

Edgar Fajardo

Reporter

Carl Edquist

Priority

Major

Fix versions

Labels

Components