+++
title = "libtool: getting rid of 180,000 sed forks"
date = 2015-03-05T08:10:36+00:00
+++
When compiling systemd on rawhide, we noticed a significant slowdown in compile time. 
Investigating further, it turns out, that libtool forks an incredible amount of sed. 
Running perf showed 30% of the "make all" was spent in bash. strace showed an execve of 
sed with the same arguments 180,000 times!!!!! 

<!-- more -->

Looking in the code reveals the culprit:
```bash
_G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;;
```
and
```bash
qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
```

`sed_quote_subst` looks like this:
```bash
sed_quote_subst='s|\([`"$\\]\)|\\\1|g'
``` 

This substitution can be easily done in bash, without forking sed. So I came up with a patch for libtool:

```diff
--- libtool.orig	2015-03-04 15:44:39.632703654 +0100
+++ libtool.fixed	2015-03-04 17:16:42.495983610 +0100
@@ -1607,7 +1607,15 @@
     while test 0 -lt $#; do
       case $1 in
         *[\\\`\"\$]*)
-	  _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;;
+          if test set = "${BASH_VERSION+set}"; then
+            _G_unquoted_arg=${1//\\/\\\\}
+            _G_unquoted_arg=${_G_unquoted_arg//\"/\\\"}
+            _G_unquoted_arg=${_G_unquoted_arg//\$/\\\$}
+            _G_unquoted_arg=${_G_unquoted_arg//\`/\\\`}
+          else
+              _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"`
+          fi
+          ;;
         *)
           _G_unquoted_arg=$1 ;;
       esac
@@ -5756,7 +5764,11 @@
   if test \"\$libtool_execute_magic\" != \"$magic\"; then
     file=\"\$0\""

-    qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
+    if test set = "${BASH_VERSION+set}"; then
+      qECHO=$(_G=$($ECHO "$ECHO");_G=${_G//\\/\\\\};_G=${_G//\"/\\\"};_G=${_G//\$/\\\$};_G=${_G//\`/\\\`};printf '%s\n' "$_G")
+    else
+      qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
+    fi
     $ECHO "\
```

With that patch, I compared the runtime of a compile (with ccache on). Original libtool:
```console
$ time make -j4
[…]
real 5m23.880s
user 11m39.731s
sys 7m12.578s
```
Fixed libtool: 
```console
$ time make -j4
[…]
real 2m47.325s
user 9m33.561s
sys 0m57.522s
```
The old version spent half of the time creating a pipe, forking sed and parsing the 
output. Someone with connections to the libtool maintainers, please point them to 
this problem! Reported as [bug 20006](http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006).