inets httpd cgi script: Why am I getting a “No such file or directory” error?

Question

When I try to request a cgi script from an inets httpd server, I get this error:

sh: /Users/7stud/erlang_programs/inets_proj/cgi-bin/cgi-bin/1.pl: 
No such file or directory

I notice that the cgi-bin component of the path is doubled. My cgi script is actually located at:

/Users/7stud/erlang_programs/inets_proj/cgi-bin/1.pl

Here is my httpd server proplist_file:

[
  {modules, [
        mod_alias,
        mod_cgi
  ]},
  { bind_address, "localhost"}, 
  {port,0},
  {server_name,"httpd_test"},
  {server_root,"."},
  {document_root,"./htdocs"},
  {script_alias, {"/cgi-bin/", "./cgi-bin/"} }
].

According to the httpd docs :

CGI Properties - Requires mod_cgi

{script_alias, {Alias, RealName}}
Alias = string() and RealName = string(). Have the same behavior as property alias, except that they also mark the target directory as containing CGI scripts. URLs with a path beginning with Alias are mapped to scripts beginning with RealName, for example:

 {script_alias, {"/cgi-bin/", "/web/cgi-bin/"}} 

Access to http://your.server.org/cgi-bin/foo would cause the server to run the script /web/cgi-bin/foo .

And:

{server_root, path()}
Defines the home directory of the server, where log files, and so on, can be stored. Relative paths specified in other properties refer to this directory.

More info:

5> httpd:info(S).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
 {server_name,"httpd_test"},
 {script_alias,{"/cgi-bin/","./cgi-bin/"}},
 {bind_address,{127,0,0,1}},
 {modules,[mod_actions,mod_alias,mod_cgi,mod_get,mod_head,
           mod_log]},
 {server_root,"."},
 {port,59641},
 {document_root,"./htdocs"}]

6> pwd().
/Users/7stud/erlang_programs/inets_proj
ok

7> ls().
cgi-bin         cl.beam         cl.erl          htdocs          
s.beam          s.erl           server.conf     
ok

Why am I getting a doubled cgi-bin component in my request url?

Answer 1

If I use a full path to the server's cgi-bin directory in the script_alias option, then I can successfully request a cgi script. I can't get a relative path to work. This is the httpd server configuration that worked for me:

server.conf:

[
  {modules, [
    mod_alias,
    mod_actions,
    mod_cgi,
    mod_get
  ]},
  {bind_address, "localhost"}, 
  {port,0},
  {server_name,"httpd_test"},
  {server_root,"/Users/7stud/erlang_programs/inets_proj"},
  {document_root,"./htdocs"},
  {script_alias, {"/cgi-bin/", "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"} }
].

Directory structure:

$ tree inets_proj
inets_proj
├── cgi-bin
│   └── 1.pl
├── cl.beam
├── cl.erl
├── htdocs
│   └── file1.txt
├── s.beam
├── s.erl
└── server.conf

s.erl:

-module(s).
-compile(export_all).

ensure_inets_start() ->
    case inets:start() of
        ok -> ok;
        {error,{already_started,inets}} -> ok
    end.

%After start(), need to lookup the port with:
%      3> httpd:info(Server)

start() ->
    ok = s:ensure_inets_start(),

    {ok, Server} = inets:start(httpd, 
        [{proplist_file, "./server.conf"}]
    ),
    Server.


stop(Server) ->
    ok = inets:stop(httpd, Server).

cgi script 1.pl (make sure the file has executable permissions!):

#!/usr/bin/env perl

use strict;
use warnings;
use 5.020;
use autodie;
use Data::Dumper;

print "Content-type: text/html\n\n";
print "Hello, Perl.\n";

file1.txt:

Hello, world!

In the shell:

$ erl
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V8.2  (abort with ^G)

1> c(s).
{ok,s}

2> S = s:start().
<0.79.0>

3> httpd:info(S).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
 {server_name,"httpd_test"},
 {script_alias,{"/cgi-bin/",
                "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"}},
 {bind_address,{127,0,0,1}},
 {modules,[mod_alias,mod_actions,mod_cgi,mod_get]},
 {server_root,"/Users/7stud/erlang_programs/inets_proj"},
 {port,54344},
 {document_root,"./htdocs"}]

4> 

In a terminal window:

~$ curl -v "http://localhost:54344/cgi-bin/1.pl"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 54344 (#0)
> GET /cgi-bin/1.pl HTTP/1.1
> Host: localhost:54344
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 03:18:05 GMT
< Server: inets/6.3.4
< Transfer-Encoding: chunked
< Content-Type: text/html
< 
Hello, Perl.
* Connection #0 to host localhost left intact


~$ curl -v "http://localhost:54344/file1.txt"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 54344 (#0)
> GET /file1.txt HTTP/1.1
> Host: localhost:54344
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 03:18:56 GMT
< Server: inets/6.3.4
< Content-Type: text/plain
< Etag: nCZT0114
< Content-Length: 14
< Last-Modified: Mon, 26 Feb 2018 02:51:52 GMT
< 
Hello, world!
* Connection #0 to host localhost left intact
$

Despite what the docs say about having to use the host syntax as output by httpd:info() --rather than what you specify for {bind_address, "localhost"} --I can use localhost in the request.

Here's an example using httpc to make a cgi post request to an httpd server:

-module(cl).
-compile(export_all).

ensure_inets_start() ->
    case inets:start() of
        ok -> ok;
        {error,{already_started,inets}} -> ok
    end.

start() ->
    ensure_inets_start().

%%   Need "http://" at start of url:
%%
%%   cl:myget("http://localhost:62049/file1.txt")
%%
%%   Need to look up port with httpd:info(Server)

myget(Url) ->
    {ok, ReqRef} = 
        httpc:request(get, {Url, []}, [], [{sync, false}]),

    receive
        {http, {ReqRef, Result}} ->
            Result
    after 2000 ->
        my_error
    end.

stop() ->
    inets:stop().

url(Port) ->
    "http://localhost:" ++ integer_to_list(Port) ++ "/cgi-bin/1.pl".

mypost(Port) ->
    Url = url(Port),
    Headers = [],
    ContentType = "application/x-www-form-urlencoded",
    Body = "a=1&b=2",

    Request = {Url, Headers, ContentType, Body},
    HttpOptions = [],
    Options = [{sync, false}],
    {ok, ReqRef} = httpc:request(post, Request, HttpOptions, Options),

    receive
        {http, {ReqRef, Result}} ->
            Result;
        Other -> Other
    after 1000 ->
        my_error
    end.

There seems to be something wrong with httpd's implementation of cgi as I am unable to make a cgi post request containing json data. A cgi script has to read the body of the request to get the raw string, and I've tried doing that with both a perl cgi script, $str = $q->param('POSTDATA'); and a python cgi script, my_dict = json.load(sys.stdin) and I can't get the body of the request from the httpd server.