initial commit

content/2015/03/05/_index.md

@@ -0,0 +1,5 @@
++++
+transparent = true
+sort_by = "date"
+paginate_by = 5
++++

content/2015/03/05/libtool-getting-rid-of-180000-sed-forks.md

@@ -0,0 +1,79 @@
++++
+title = "libtool: getting rid of 180,000 sed forks"
+date = 2015-03-05T08:10:36+00:00
++++
+When compiling systemd on rawhide, we noticed a significant slowdown in compile time. 
+Investigating further, it turns out, that libtool forks an incredible amount of sed. 
+Running perf showed 30% of the "make all" was spent in bash. strace showed an execve of 
+sed with the same arguments 180,000 times!!!!! 
+
+<!-- more -->
+
+Looking in the code reveals the culprit:
+```bash
+_G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;;
+```
+and
+```bash
+qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
+```
+
+`sed_quote_subst` looks like this:
+```bash
+sed_quote_subst='s|\([`"$\\]\)|\\\1|g'
+``` 
+
+This substitution can be easily done in bash, without forking sed. So I came up with a patch for libtool:
+
+```diff
+--- libtool.orig	2015-03-04 15:44:39.632703654 +0100
++++ libtool.fixed	2015-03-04 17:16:42.495983610 +0100
+@@ -1607,7 +1607,15 @@
+     while test 0 -lt $#; do
+       case $1 in
+         *[\\\`\"\$]*)
+-	  _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;;
++          if test set = "${BASH_VERSION+set}"; then
++            _G_unquoted_arg=${1//\\/\\\\}
++            _G_unquoted_arg=${_G_unquoted_arg//\"/\\\"}
++            _G_unquoted_arg=${_G_unquoted_arg//\$/\\\$}
++            _G_unquoted_arg=${_G_unquoted_arg//\`/\\\`}
++          else
++              _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"`
++          fi
++          ;;
+         *)
+           _G_unquoted_arg=$1 ;;
+       esac
+@@ -5756,7 +5764,11 @@
+   if test \"\$libtool_execute_magic\" != \"$magic\"; then
+     file=\"\$0\""
+
+-    qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
++    if test set = "${BASH_VERSION+set}"; then
++      qECHO=$(_G=$($ECHO "$ECHO");_G=${_G//\\/\\\\};_G=${_G//\"/\\\"};_G=${_G//\$/\\\$};_G=${_G//\`/\\\`};printf '%s\n' "$_G")
++    else
++      qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
++    fi
+     $ECHO "\
+```
+
+With that patch, I compared the runtime of a compile (with ccache on). Original libtool:
+```console
+$ time make -j4
+[…]
+real 5m23.880s
+user 11m39.731s
+sys 7m12.578s
+```
+Fixed libtool: 
+```console
+$ time make -j4
+[…]
+real 2m47.325s
+user 9m33.561s
+sys 0m57.522s
+```
+The old version spent half of the time creating a pipe, forking sed and parsing the 
+output. Someone with connections to the libtool maintainers, please point them to 
+this problem! Reported as [bug 20006](http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006).