Notes & Trouble Shooting¶

This is a place to put in odds and ends that we think we may want to include in the book.

design goals¶

everything built from source using a bootstrap system that has at least a minimal GNU development environment
without any build dependencies other than a C development toolchain
including instructions using modern versions of the toolchain components

cross-platform and implications of upstream binaries¶

FreeSA book should mention that since we are building for MIPS this really highlights the need for no dependency upon upstream x86 binaries

... Like will it be possible to build Java on MIPS? ....

Part of what we like about moving to new platforms is that it really forces questions about "how from source is this stuff?" and hopefully clearly exposes otherwise non-obvious binary dependencies

h2. mount and df
by default the mount and df were giving eric very confusing results in the chrooted environment until this change was made:

root:/# rm /etc/mtab
root:/# ln -s /proc/mounts /etc/mtab

Testing a Sysroot compiler¶

Or, "What do you mean, './hello: No such file or directory'?"

A compiler built using $SYSROOT will not produce a functional binary for hello world, unless we include the -static option to gcc

bob:/tmp$ mipsel-unknown-linux-uclibc-gcc hello.c -o hello
bob:/tmp$ ./hello 
-su: ./hello: No such file or directory
bob:/tmp$

What? It's right there:

bob:/tmp$ ls -l hell*   
-rwxrwxr-x 1 bob bob 5783 Dec 12 16:38 hello
-rw-rw-r-- 1 bob bob   75 Dec 12 16:35 hello.c
bob:/tmp$

But, that isn't what it means. This is a dynamic linking problem.

bob:/tmp$ file ./hello
./hello: ELF 32-bit LSB executable, MIPS, MIPS32 version 1 (SYSV), dynamically linked (uses shared libs), not stripped
bob:/tmp$ readelf -a hello | grep lib
      [Requesting program interpreter: /lib/ld-uClibc.so.0]
 0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 0x00000001 (NEEDED)                     Shared library: [libc.so.0]
    14: 00400840     0 FUNC    GLOBAL DEFAULT  UND +uClibc_main
    55: 00400840     0 FUNC    GLOBAL DEFAULT  UND +uClibc_main
  000000: Version: 1  File: libgcc_s.so.1  Cnt: 1

Note the /lib/ld-uClibc.so.0 and that only exists in our sysroot.

bob:/tmp$ find / -name ld-uClibc.so.0
/opt/uclibc/sysroot/lib/ld-uClibc.so.0
/opt/uclibc_build/uclibc/lib/ld-uClibc.so.0
bob:/tmp$

However, static linking in GCC is a snap:

bob:/tmp$ mipsel-unknown-linux-uclibc-gcc -static hello.c -o hello
bob:/tmp$ ./hello 
Hello World!
bob:/tmp$

There! That did what we wanted.

And we can see that it's statically linked:

bob:/tmp$ file ./hello
./hello: ELF 32-bit LSB executable, MIPS, MIPS32 version 1 (SYSV), statically linked, not stripped

(You can run it, too, but it's kind of boring.)

watch¶

man watch
watch - execute a program periodically, showing output fullscreen

watch might be a better alternative than this sort of while-true-sleep loop:

while true; do sleep 5; clear; ls -ltr /opt/uclibc_build/Logs; tail /opt/uclibc_build/Logs/$(ls -tr /opt/uclibc_build/Logs | tail -n1); done

Screen¶

person 1:

$ screen
ctrl-A, :, multiuser on <CR>
ctrl-A, :, addacl [login of person 2]

person 2:

$ screen -ls [person 1]/

produces something like:

There is a suitable screen on:
    9734.freesa (Multi, attached)

then

$ screen -r [person 1]/9734.freesa

uClibc multiple definition of compare_and_swap¶

ln -fs sgidefs.h ./include/
  LD libuClibc-0.9.29.so
libc/libc_so.a(libc_pthread_init.oS): In function `testandset':
libc_pthread_init.c:(.text+0x0): multiple definition of `testandset'
libc/libc_so.a(forward.oS):forward.c:(.text+0x0): first defined here
libc/libc_so.a(libc_pthread_init.oS): In function `__compare_and_swap':
libc_pthread_init.c:(.text+0x24): multiple definition of `__compare_and_swap'
libc/libc_so.a(forward.oS):forward.c:(.text+0x24): first defined here
make[1]: *** [lib/libc.so] Error 1
make: *** [lib/libc.so.0] Error 2

Resolved with Freesa/patches/uclibc-0.9.29-fix-pthread-mips-build-1.patch

TroubleShooting BusyBox Building¶

There are a number of subtle, but important differences between uClibc 0.9.29 and 0.9.30 which make for interesting compiler errors, if uClibc 0.9.30 is build using simply make oldconfig starting with a perfectly functional 0.9.29 .config:

  AR      util-linux/volume_id/lib.a
  LINK    busybox_unstripped
Trying libraries: crypt m
Failed: -Wl,--start-group -lcrypt -lm -Wl,--end-group
Output of:
mipsel-unknown-linux-uclibc-gcc -Wall -Wshadow -Wwrite-strings -Wundef -Wstrict-prototypes -Wunused -Wunused-parameter -Wmissing-prototypes -Wmissing-declarations -Wdeclaration-after-statement -Wold-style-definition -fno-builtin-strlen -finline-limit=0 -fomit-frame-pointer -ffunction-sections -fdata-sections -fno-guess-branch-probability -funsigned-char -static-libgcc -falign-functions=1 -falign-jumps=1 -falign-labels=1 -falign-loops=1 -Os -static -o busybox_unstripped -Wl,--sort-common -Wl,--sort-section,alignment -Wl,--gc-sections -Wl,--start-group applets/built-in.o archival/lib.a archival/libunarchive/lib.a console-tools/lib.a coreutils/lib.a coreutils/libcoreutils/lib.a debianutils/lib.a e2fsprogs/lib.a editors/lib.a findutils/lib.a init/lib.a libbb/lib.a libpwdgrp/lib.a loginutils/lib.a mailutils/lib.a miscutils/lib.a modutils/lib.a networking/lib.a networking/libiproute/lib.a networking/udhcp/lib.a printutils/lib.a procps/lib.a runit/lib.a selinux/lib.a shell/lib.a sysklogd/lib.a util-linux/lib.a util-linux/volume_id/lib.a archival/built-in.o archival/libunarchive/built-in.o console-tools/built-in.o coreutils/built-in.o coreutils/libcoreutils/built-in.o debianutils/built-in.o e2fsprogs/built-in.o editors/built-in.o findutils/built-in.o init/built-in.o libbb/built-in.o libpwdgrp/built-in.o loginutils/built-in.o mailutils/built-in.o miscutils/built-in.o modutils/built-in.o networking/built-in.o networking/libiproute/built-in.o networking/udhcp/built-in.o printutils/built-in.o procps/built-in.o runit/built-in.o selinux/built-in.o shell/built-in.o sysklogd/built-in.o util-linux/built-in.o util-linux/volume_id/built-in.o -Wl,--end-group -Wl,--start-group -lcrypt -lm -Wl,--end-group
==========
/opt/uclibc/lib/gcc/mipsel-unknown-linux-uclibc/4.3.2/../../../../mipsel-unknown-linux-uclibc/bin/ld: cannot find -lcrypt
collect2: ld returned 1 exit status
make: *** [busybox_unstripped] Error 1
bob:/opt/bbox/busybox-1.13.1$

First looking at the host system we see:

root:/usr/lib# ldconfig -p | grep crypt
    libcrypto.so.0.9.8 (libc6) => /usr/lib/libcrypto.so.0.9.8
    libcrypto.so (libc6) => /usr/lib/libcrypto.so
    libcrypt.so.1 (libc6, OS ABI: Linux 2.6.0) => /lib/libcrypt.so.1
    libcrypt.so (libc6, OS ABI: Linux 2.6.0) => /usr/lib/libcrypt.so
root:/usr/lib# ls -l *crypt*so*
lrwxrwxrwx 1 glibc   glibc        23 May 18  2008 libcrypt.so -> ../../lib/libcrypt.so.1
lrwxrwxrwx 1 openssl openssl      18 Aug  6 07:38 libcrypto.so -> libcrypto.so.0.9.8
-r-xr-xr-x 1 openssl openssl 1792211 Aug  6 07:38 libcrypto.so.0.9.8
root:/usr/lib#
root:/usr/lib# ls -l ../../lib/libcrypt.so.1
lrwxrwxrwx 1 glibc glibc 15 May 18  2008 ../../lib/libcrypt.so.1 -> libcrypt-2.7.so
root:/usr/lib# ls -l ../../lib/libcrypt-2.7.so
-rwxr-xr-x 1 glibc glibc 44072 Aug  4 19:44 ../../lib/libcrypt-2.7.so
root:/usr/lib#

Now looking at the $SYSROOT we see the libraries are missing:

/opt/uclibc/bin:/tmp/bob/bin:/bin:/usr/bin
bob:/opt/bbox/busybox-1.13.1$ find /opt -name "*crypt*so*" 
bob:/opt/bbox/busybox-1.13.1$

Looking in the output logs of the build we see:

$ grep crypt *
step03.txt:  CLEAN libcrypt
step03.txt:# Remove crypt.h since libcrypt was disabled upon request
step03.txt:rm -f /opt/uclibc/sysroot/usr/include/crypt.h
step05.txt:# Remove crypt.h since libcrypt was disabled upon request
step05.txt:rm -f /opt/uclibc/sysroot/usr/include/crypt.h
step07.txt:# Remove crypt.h since libcrypt was disabled upon request
step07.txt:rm -f /opt/uclibc/sysroot/usr/include/crypt.h

Looking at the uClibc .config we see

# UCLIBC_HAS_CRYPT_IMPL is not set

That was at issue. Also, this is a problem: # UCLIBC_HAS_SYSLOG is not set.

Also available in: HTML TXT

FreeSA

Wiki