减少bash环境变量的大小

Question

Let's say I have a program that takes a very large number of command line arguments or a very large single command line argument. Let's assume, that my application wants to accept a single command line argument as a string of 65536 symbols (64kB). As everybody knows there is a limit on what amount of bytes one can pass to process with a command line and environment variables.

And here is a little experiment:

$ set > t
$ ls -lF | grep "\<t\>"
-rw-r--r--  1 sergey sergey 118737 Oct 14 23:45 t

And this 118737 bytes is about 115kB. Header file /usr/include/linux/limits.h says that ARG_MAX is 131072 which is exactly 128kB.

Is there a way to reduce environment variables size of user bash session?

Answer 1

set | wc -c set | wc -c (which is a much simpler way to count the size of all shell variables) is not relevant, since most of those variables are not exported and the limit applies only to exported variables and command line arguments.

Try export | wc -c export | wc -c to get a realistic view. On one shell session I happen to have open:

$ set | wc -c
241235
$ export | wc -c
4652

You can also get a variety of useful information from xargs :

$ xargs --show-limits </dev/null
Your environment variables take up 3730 bytes
POSIX upper limit on argument length (this system): 2091374
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2087644
Size of command buffer we are actually using: 131072

Or a simpler report from getconf :

$ getconf ARG_MAX
2097152

It is quite possible that your particular system allows more than ARG_MAX bytes.

Answer 2

I believe the export | wc -c export | wc -c command is not enough if you want to use that route as an approximation. I use bash shell functions. A LOT of them. And I export them to be available in subshells. Those are only discovered with export -f . So in theory you really need the output of both.

However, looking at my machine:

$  export -p | wc && export -f | wc              
    147     479    9309
   6940   17029  150917

$  xargs --show-limits </dev/null                        
Your environment variables take up 121160 bytes
POSIX upper limit on argument length (this system): 1973944
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 1852784
Size of command buffer we are actually using: 131072

Perhaps the output of the export command gives a good look at the ratio between exported variables and functions, but it isn't reflective of the actual space used (9309 + 150917 != 121160) .

And that kind of makes sense. Here is the output of one random function in my environment, via export -f :

usage_init () 
{ 
    USAGE_OPT_P1="-p pathvar - The envvar to process. Default is";
    USAGE_OPT_P2="             PATH, if -p is not specified.";
    USAGE_OPT_P3="             The '-p' is optional, in which case";
    USAGE_OPT_P4F="             the first ARGUMENT is assumed to be";
    USAGE_OPT_P4O="             the ARGUMENT is assumed to be";
    USAGE_OPT_P5="             the pathvar.";
    USAGE_OPT_S="-s sep - Path element separator. Defaults to ':'.";
    USAGE_OPT_X="-x - Exports 'pathvar'.";
    USAGE_OPT_V="-v - Executes function 'listpath' after assignment.";
    USAGE_OPT_H="-h - Gives usage message.";
    USAGE_OPT_E="-e - 'pathspec' is interpreted as an egrep regexp.";
    USAGE_OPT_N="-n - Add index numbers to the results."
}
declare -fx usage_init

Here is the same function plucked out of env

BASH_FUNC_usage_init%%=() {  USAGE_OPT_P1="-p pathvar - The envvar to process. Default is";
 USAGE_OPT_P2="             PATH, if -p is not specified.";
 USAGE_OPT_P3="             The '-p' is optional, in which case";
 USAGE_OPT_P4F="             the first ARGUMENT is assumed to be";
 USAGE_OPT_P4O="             the ARGUMENT is assumed to be";
 USAGE_OPT_P5="             the pathvar.";
 USAGE_OPT_S="-s sep - Path element separator. Defaults to ':'.";
 USAGE_OPT_X="-x - Exports 'pathvar'.";
 USAGE_OPT_V="-v - Executes function 'listpath' after assignment.";
 USAGE_OPT_H="-h - Gives usage message.";
 USAGE_OPT_E="-e - 'pathspec' is interpreted as an egrep regexp.";
 USAGE_OPT_N="-n - Add index numbers to the results."
}

export (and I think type ) appear to do some prettying for presentation which inflates the wc -c value.