82 lines
2.5 KiB
Markdown
82 lines
2.5 KiB
Markdown
+++
|
|
title = "libtool: getting rid of 180,000 sed forks"
|
|
date = 2015-03-05T08:10:36+00:00
|
|
[taxonomies]
|
|
tags = ["fedora", "programming"]
|
|
+++
|
|
When compiling systemd on rawhide, we noticed a significant slowdown in compile time.
|
|
Investigating further, it turns out, that libtool forks an incredible amount of sed.
|
|
Running perf showed 30% of the "make all" was spent in bash. strace showed an execve of
|
|
sed with the same arguments 180,000 times!!!!!
|
|
|
|
<!-- more -->
|
|
|
|
Looking in the code reveals the culprit:
|
|
```bash
|
|
_G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;;
|
|
```
|
|
and
|
|
```bash
|
|
qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
|
|
```
|
|
|
|
`sed_quote_subst` looks like this:
|
|
```bash
|
|
sed_quote_subst='s|\([`"$\\]\)|\\\1|g'
|
|
```
|
|
|
|
This substitution can be easily done in bash, without forking sed. So I came up with a patch for libtool:
|
|
|
|
```diff
|
|
--- libtool.orig 2015-03-04 15:44:39.632703654 +0100
|
|
+++ libtool.fixed 2015-03-04 17:16:42.495983610 +0100
|
|
@@ -1607,7 +1607,15 @@
|
|
while test 0 -lt $#; do
|
|
case $1 in
|
|
*[\\\`\"\$]*)
|
|
- _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;;
|
|
+ if test set = "${BASH_VERSION+set}"; then
|
|
+ _G_unquoted_arg=${1//\\/\\\\}
|
|
+ _G_unquoted_arg=${_G_unquoted_arg//\"/\\\"}
|
|
+ _G_unquoted_arg=${_G_unquoted_arg//\$/\\\$}
|
|
+ _G_unquoted_arg=${_G_unquoted_arg//\`/\\\`}
|
|
+ else
|
|
+ _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"`
|
|
+ fi
|
|
+ ;;
|
|
*)
|
|
_G_unquoted_arg=$1 ;;
|
|
esac
|
|
@@ -5756,7 +5764,11 @@
|
|
if test \"\$libtool_execute_magic\" != \"$magic\"; then
|
|
file=\"\$0\""
|
|
|
|
- qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
|
|
+ if test set = "${BASH_VERSION+set}"; then
|
|
+ qECHO=$(_G=$($ECHO "$ECHO");_G=${_G//\\/\\\\};_G=${_G//\"/\\\"};_G=${_G//\$/\\\$};_G=${_G//\`/\\\`};printf '%s\n' "$_G")
|
|
+ else
|
|
+ qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"`
|
|
+ fi
|
|
$ECHO "\
|
|
```
|
|
|
|
With that patch, I compared the runtime of a compile (with ccache on). Original libtool:
|
|
```console
|
|
$ time make -j4
|
|
[…]
|
|
real 5m23.880s
|
|
user 11m39.731s
|
|
sys 7m12.578s
|
|
```
|
|
Fixed libtool:
|
|
```console
|
|
$ time make -j4
|
|
[…]
|
|
real 2m47.325s
|
|
user 9m33.561s
|
|
sys 0m57.522s
|
|
```
|
|
The old version spent half of the time creating a pipe, forking sed and parsing the
|
|
output. Someone with connections to the libtool maintainers, please point them to
|
|
this problem! Reported as [bug 20006](http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006).
|