Jon Jensen
End Point Corporation
endpointdev.com
YAPC::Europe, Kiev, Ukraine
12 August 2013
(See also video of this talk.)
That’s what I’ll focus on here
yum install 'perl(DBD::Pg)'
Actual package name was
perl-DBD-Pg
But this fails:
yum install perl-HTTP-Date
Don’t need to know package or parent module name:
yum install 'perl(HTTP::Date)'
Installs perl-libwww-perl which provides it.
An RPM that requires
everything you need,
including specific versions
where necessary.
Install it all with one command:
yum install myapp-deps
Changed requirements between
myapp-deps-1.3 and myapp-deps-1.4?
Roll back may work:
yum downgrade myapp-deps-1.3
RHEL has a reputation of not having
many CPAN modules available packaged.
Not true with EPEL, and in
Debian there are plenty in the main repo.
Of course if the one CPAN module you want
happens not to be packaged yet, it’s all vain.
What version of Perl?
And CPAN module versions?
The whole point is long-term support.
Without updates breaking things.
(Debian is not a long-term support distro,
but it’s widely used, so I show it for comparison.)
| OS | Released | Perl | EOL |
|---|---|---|---|
| RHEL 5 | 2007 | 5.8.8 | 2017 |
| RHEL 6 | 2010 | 5.10.1 | 2020 |
| Ubuntu 10.04 LTS | 2010 | 5.10.1 | 2015 |
| Debian 6 “squeeze” | 2011 | 5.10.1 | 2014 |
| Ubuntu 12.04 LTS | 2012 | 5.14.2 | 2017 |
| Debian 7 “wheezy” | 2013 | 5.14.2 | ? |
Perl releases now more frequent.
Distros not keeping up.
OpenBSD 5.3 was released last month.
It includes Perl 5.12.2.
Yes, it’s possible.
Some system packages
depend on perl.
Which ones?
repoquery --whatrequires perl
apt-cache rdepends perl
That’s the default when building from source.
Simple, easy, cheap (in the moment).
If only ever this one server
running one Perl app
(or a few that use the
same libraries), this will work.
I’ll do the same thing
on each server!
Install a CPAN module on one server during development.
When you later install on production,
you forget to install it there.
Hurry and install it, but the cpan(m) client installs
a different (newer) version from CPAN.
Maybe that matters now, maybe later, maybe not at all.
If the exact same distro, version, and architecture,
you can just clone /usr/local over.
Very nice, and solve some
of these problems.
Cache CPAN tarballs and thus avoid install-time
dependency on public Internet & CPAN.
If you install needed system libraries from source,
things get even more complicated to reproduce.
But I’m talking here about server environments.
Need reliable, supportable, reproducible over years.
We needed:
We also needed many, many
CPAN modules, and wanted their
versions frozen.
RHEL 5: local-perl 5.10.0 and over 700 CPAN packages
RHEL 6: local-perl 5.14.1 and over 300 CPAN packages
In /usr/local/bin/perl so it’s in default PATH already.
Going back to `yum install myapp-deps`
— now that can install our custom perl
and all the CPAN modules.
It is, once it’s done.
Let’s see what it takes to get there.
%HOME %{expand:%%(cd; pwd)}
%_topdir %{HOME}/rpm
%debug_package %nil
%_signature gpg
%_gpg_name hosting@endpointdev.com
%packager End Point Hosting Team
%vendor End Point Corporation https://packages.endpointdev.com/
%dist .ep
%__perl_provides %{HOME}/bin/local-perl.prov
%__perl_requires %{HOME}/bin/local-perl.req
%__find_provides %{HOME}/bin/find-provides
%__find_requires %{HOME}/bin/find-requires
%_prefix /usr/local
%_exec_prefix %{_prefix}
%_bindir %{_exec_prefix}/bin
%_sbindir %{_exec_prefix}/sbin
%_libexecdir %{_exec_prefix}/libexec
%_datadir %{_prefix}/share
%_sysconfdir %{_prefix}/etc
%_sharedstatedir %{_prefix}/com
%_localstatedir %{_prefix}/var
%_lib lib64
%_libdir %{_exec_prefix}/%{_lib}
%_oldlibdir /usr/lib64
%_includedir %{_prefix}/include
%_oldincludedir /usr/include
%_infodir %{_prefix}/info
%_mandir %{_datadir}/man
%__perl /usr/local/bin/perl
We use cpan2rpm.
Ours is customized and called
local-cpan2rpm to support our
nonstandard installation location
and package naming convention.
Details of using cpan2rpm are too much
to cover in this talk.
Other similar tools:
Watch for:
Take RHEL’s source RPM and adapt for
local-perl location and naming, and
whatever other changes you’ll make.
The prominence of x86_64 is nice.
We aren’t supporting RHEL 6 32-bit.
Could automate but arch often requires manual attention.
Build a “Yum repo RPM”, which adds support
for your custom yum repository.
Easier installation, upgrades, removal.
We wanted to be kinda lazy and
take advantage of the fact that
/usr/local/bin was usually in the path,
so if we installed perl there, it would work.
We no longer think that’s so important,
since you still have to make the shebang line
point to /usr/local/bin/perl.
Could use the alternatives system borrowed
from Debian and available in RHEL.
EPEL did this with python26
alongside system python (2.4).
Usual filesystem locations, different names.
Just add your weird /opt/$your_org/perl-$version/bin
manually to PATH.
Because we now need multiple custom perls on a machine too.
We plan to do that for RHEL/CentOS 7.
Watch out using [/usr]/bin/env.
Better to hardcode to a specific perl.
Or use nonversioned wrappers or shims.
It’s easy to lose your custom PATH, e.g. on re-exec.
Check out Red Hat Software Collections
and Fedora Software Collections.
Brings to RHEL 6 Perl 5.16.3
installed in /opt/rh/perl516.
Includes other newer software:
PHP, Python, Ruby, databases.
Still called a beta.
It would be good to join forces with Fedora on this.
Then you don’t do all the work or hosting yourself.
OTOH I doubt they would want our unthreaded perl.
We were on our own with that.
We’ll be looking to Software Collections for
better ways to do our custom packaging.
Except maybe in site_perl path.
Examples: Module::Build or Sys::Syslog
core module conflict.
Even if we can work under the umbrella of a group,
such as EPEL, Software Collections, or some other repo,
will a new package be available fast enough for
the developer who needs it?
Developers have little patience waiting for
you to build packages for CPAN modules.
Can’t I just install this one from source?
And experimentation speed is important.
May try several CPAN modules
before finding the right one.
Why package those that weren’t selected?
We’re starting to recommend developers
use local::lib for experiments,
and we only build RPMs once they’re
needed for production.
At least business software is.
So this is all worth the bother.
Ideal would be to have RPM-installed CPAN modules
of any version simultaneously installable,
that Carton can selectively make visible to your app.
Initial experiment failed; will try more later.
There are analogous problems, techniques,
trade-offs, and tools for everything I’ve
talked about for Python, Ruby, and PHP.