Compiling Perl on Android

At YAPC::EU 2012, I only had my Asus TF101, and I wanted to play with some of the modules that the speakers were talking about.

The Google-provided "Perl for Android" (part of the Android Scripting Environment) is nearly useless, since you don't get the whole Perl core, the CPAN shell does not work, and you don't have a compiler for XS modules.

Another option would be to cross-compile from a normal computer, but apart from the fact that I did not another computer with me, you still would not be able to install XS modules directly from the cross-compiled Perl.

So I did what any self-respecting programmer would do: tried to build my own. I haven't yet managed to completely build the Perl distribution, but I have come some way, and I thought that it would be nice to write things down.

Prerequisites

First of all, you need root access on your device: we have to create filesystems, change owners and permissions, mount images.

Then, you need quite some RAM, storage, and processor. I wouldn't try it on a phone or a low-powered tablet.

Finally, you need busybox: the bare-bones command line tools shipped with most Android images are so limited that not even the very conservative Perl Configure can use them. You'll want to execute this (or something similar) to get busybox binaries in preference to the built-in ones:

export PATH=/system/xbin:$PATH

Preparations

We need a native compiler. I found a well-built gcc on a wiki about R: http://rwiki.sciviews.org/doku.php?id=getting-started:installation:android

Get the two gcc tarballs.

We're going to build a filesystem image, mount it via the loopback device, and install everything there: this way, the image can be moved between devices (we'll make sure everything is as relocatable as possible). I stored everything under /data/media/native (I created that directory). /data/media is, on my device, the actual store that is exposed (via a sort of bind-mount) as /sdcard, with the difference that the latter has noexec, the former doesn't.

We want a rather large image, let's say 3 GB. I tried to make a sparse file with dd if=/dev/zero of=native.e2.diskimg bs=1024 seek=3000000 count=1 but it failed. I ended up creating two smaller files and concatenating them. Then:

# cd /data/media/native

# mke2fs -f native.e2.diskimg

# cat >  /data/media/native/start <<EOF
#!/xbin/ash
export PATH=/system/xbin:$PATH
BASE="$(dirname "$(readlink -f "$0")")"
DEST=/Removable
if [[ -d $DEST/fs/bin ]]; then
   true
else
   mkdir -p $DEST/fs
   mount -o loop -t ext2 $BASE/native.e2.diskimg $DEST/fs
fi
echo PATH=\"$DEST/fs/bin:/system/xbin:$PATH\"
echo LD_LIBRARY_PATH=\"$DEST/fs/lib\"
EOF

# cat >  /data/media/native/stop <<EOF
#!/xbin/ash
export PATH=/system/xbin:$PATH
BASE="$(dirname "$(readlink -f "$0")")"
DEST=/Removable
if [[ -d $DEST/fs/bin ]]; then
   umount $DEST/fs
else
   true
fi
EOF

# chmod +x start stop

Let's mount the (still empty) image (as a normal user, not root):

$ eval $(su -c /data/media/native/start)

and fill it up:

# cd /Removable/fs
# bzcat2 $wherever/android_gcc_r2a.tar.bz2|tar xvf -
# bzcat2 $wherever/android_gcc_supplement.tar.bz2|tar xvf -

(my tar does not understand bzip-ed archives, so I had to decompress them separately)

We need some temporary directories:

# mkdir -p /Removable/fs/tmp
# chown root:root /Removable/fs/tmp
# chmod 1777 /Removable/fs/tmp

My shell (the built-in /bin/sh) needs a temporary space under /sqlite_stmt_journals for its "here-docs", but my image did not mount it, so I had to do it manually:

# mkdir /sqlite_stmt_journals
# mount -t tmpfs none /sqlite_stmt_journals

My installation also lacked a true binary, which is needed by lots of makefiles. I created it this way:

# cat > /Removable/bin/true <<EOF
#!/bin/sh
EOF
# chmod +x /Removable/bin/true

Building perl

From here on, we can run as a normal user.

Get a perl source tarball, and expand it:

$ cd /Removable/fs/tmp
$ tar zxf $perl_tarball
$ cd perl-*

We're now ready to build:

$ BASE=/Removable/fs ./Configure -de -Dprefix=$BASE \
  -Duserelocatableinc -Dcc=gcc -Dlibpth=$BASE/lib

At the end of the configuration (which should run without errors), you'll still not be able to just run make: we have to edit a few files.

First, in perl.h, just before # endif /* !USE_LOCALE_NUMERIC */, add this line:

#define IN_LOCALE_COMPILETIME 0

In the source file for Cwd, we have to tell it to look for the pwd binary all over the $PATH, or at least in /system/xbin/pwd, otherwise it won't find it, and return empty strings.

In the source file for Errno, we have to add /Removable/fs/include to the list of places where it looks for errno.h. It should probably ask the compiler for the search list, but different compilers do it differently, and it's a mess anyway.

Now, running make should build miniperl, some core libraries, perl, some more libraries, and fail as soon as XSLoader is needed. I'm still investigating this.

Future work

  • get to the end of the build
  • package the image and make it available
  • submit patches to reduce the need to muck about with paths and headers

UPDATE

Other people have been working on this, putting in way more effort, and getting better results: the master branch of perl now can be compiled (natively or cross) on Android: https://github.com/Perl/perl5/commit/b373396993794bd428d71821e0a3f89d22a2ccb0

DatesCreated: 2012-09-01 09:19:00 Last modification: 2014-02-09 14:16:14