setlocale() doesn't read from bindtextdomain() dir C++

Question

I'm testing gettext() to change the language of a simple "hello, world." string. I generated the,pot and changed. .po and .mo for 2 languages ja_JP and pt_PT . For pt_PT it worked well but ja_JP when I change the env var LANG and call setlocale() it tries to read a diff file from it was supposed. The point is to change the language without messing with the locale command, in this case locale -a doesn't have ja_JP . Programs like inkscape allow the user to change the language without doing it.

Test code:

#include <iostream>
#include <libintl.h>

std::string mygetenv(const char* var) {
    std::string rv;
    char * ptr = std::getenv(var);
    if(ptr) rv = ptr;
    return rv;
}

int main(int argc, char *argv[])
{
    std::string pwd = mygetenv("PWD");
    char* l = getenv("LANG");
    std::cout << "getenv(LANG): " << (l ? l : "NULL") << std::endl;
    char* s = setlocale(LC_ALL, "");
    std::cout << "setlocale(): " << (s ? s : "NULL") << std::endl;
    std::cout << "textdomain(): " << textdomain("gt") << "\n" << std::endl;
    pwd.append("/locales");
    std::cout << "bindtextdomain(): " << bindtextdomain("gt", pwd.c_str()) << std::endl;
    
    bind_textdomain_codeset("gt", "UTF-8");
    
    std::cout << "Default: " << gettext("hello, world!") << std::endl;
    setenv("LANG", "ja_JP.UTF-8", 1);
    s = setlocale(LC_ALL, "");
    std::cout << "setlocale(): " << (s ? s : "NULL") << std::endl;
    std::cout << "ja_JP: " << gettext("hello, world!") << std::endl;
    setenv("LANG", "pt_PT.UTF-8", 1);
    s = setlocale(LC_ALL, "");
    std::cout << "setlocale(): " << (s ? s : "NULL") << std::endl;
    std::cout << "pt_PT: " << gettext("hello, world!") << std::endl;
    return 0;
}

output:

getenv(LANG): en_US.UTF-8
setlocale(): LC_CTYPE=pt_PT.UTF-8;LC_NUMERIC=pt_PT.UTF-8;LC_TIME=pt_PT.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=pt_PT.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=pt_PT.UTF-8;LC_NAME=pt_PT.UTF-8;LC_ADDRESS=pt_PT.UTF-8;LC_TELEPHONE=pt_PT.UTF-8;LC_MEASUREMENT=pt_PT.UTF-8;LC_IDENTIFICATION=pt_PT.UTF-8
bindtextdomain(): /home/<USER>/projects/gettext_test/locales
textdomain(): gt

Default: hello, world!
setlocale(): NULL
ja_JP: hello, world!
setlocale(): pt_PT.UTF-8
pt_PT: ola, mundo!

And to check for which files it's looking for I used strace .

pt_PT:

openat(AT_FDCWD, "/home/<USER>/projects/gettext_test/locales/pt_PT.UTF-8/LC_MESSAGES/gt.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/<USER>/projects/gettext_test/locales/pt_PT.utf8/LC_MESSAGES/gt.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/<USER>/projects/gettext_test/locales/pt_PT/LC_MESSAGES/gt.mo", O_RDONLY) = 3
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3
pt_PT: ola, mundo!

ja_JP:

openat(AT_FDCWD, "/usr/lib/locale/ja_JP.UTF-8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/ja_JP.utf8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/ja_JP/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/ja.UTF-8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/ja.utf8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/ja/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/ja_JP.UTF-8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/ja_JP.utf8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/ja_JP/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/ja.UTF-8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/ja.utf8/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/ja/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
ja_JP: hello, world!

For some reason it doesn't try the locale folder in the project dir.

Dir structure with link to files:

    <project>/
        |- locales/
            |- [gt.pot][1]
            |- ja_JP/
                |- [ja.pot][2]
                |- [ja.po][3]
                |- LC_MESSAGES/
                    |- [gt.mo][4]
            |- pt_PT/
                |- [pt.pot][5]
                |- [pt.po][6]
                |- LC_MESSAGES/
                    |- [gt.mo][7]

1. http://ix.io/2Hj6
2. http://ix.io/2Hjc
3. http://ix.io/2Hj7
4. http://ix.io/2Hjd
5. http://ix.io/2Hjg
6. http://ix.io/2Hje
7. http://ix.io/2Hjh

I can't uderstand why pt_PT works but ja_JP doesn't

EDIT: adding makefile I used

all:
    g++ -o gt main.cpp

gen-template:
    xgettext -d gt -o gt.pot main.cpp -p ./locales

gen-po:
    sed --in-place ./locales/pt_PT/pt.pot --expression='s/CHARSET/UTF-8/'
    msginit -l pt_PT -o ./locales/pt_PT/pt.po -i ./locales/pt_PT/pt.pot
    sed --in-place ./locales/ja_JP/ja.pot --expression='s/CHARSET/UTF-8/'
    msginit -l ja_JP -o ./locales/ja_JP/ja.po -i ./locales/ja_JP/ja.pot

gen-mo:
    msgfmt ./locales/pt_PT/pt.po -o ./locales/pt_PT/LC_MESSAGES/gt.mo 
    msgfmt ./locales/ja_JP/ja.po -o ./locales/ja_JP/LC_MESSAGES/gt.mo 

update-pot:
    msgmerge ./locales/pt_PT/pt.po ./locales/pt_PT/pt.pot -o ./locales/pt_PT/pt.po
    msgmerge ./locales/ja_JP/ja.po ./locales/ja_JP/ja.pot -o ./locales/ja_JP/ja.po

EDIT: It works on Windows! but can't put it to work on Linux (using Ubuntu rn)

EDIT: added output locale -a :

$ locale -a

C
C.UTF-8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IL
en_IL.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
POSIX
pt_PT.utf8
.utf8

Answer 1

You cannot get ja_JP to work, when the locale ja_JP is not installed. When you look at the gettext source code you will see that it does a call setlocale("ja_JP", NULL) to determine the currently set locale. In your case, this call returns NULL and therefore no catalog ( .mo file) at all is loaded.

The intl library does not check any environment variable. The setlocale() function checks the environment, when you call it with an empty string as the second argument. That is then called the "native" environment. But since you don't have ja_JP installed, switching the locale fails, and therefore subsequent calls to gettext() do not "know" the language you were trying to select.

The rationale behind that is that your code and all linked libraries use the same language and(.) encoding. It should be prevented that for example the libc messages are in pt_PT using iso-8859-1 and your program code outputs messages in ja_JP and UTF-8.

You can actually simplify your test code and instead of setting environment variables you can just hardcode the locale with setlocale(LC_ALL, "ja_JP.UTF-8") . It has the same effect and makes it even clearer that your code cannot work as expected when the Japanese locale is not installed.

The strace output for ja_JP is most likely not caused by your code but by some library function. Compare the flags in the second argument to openat() . For pt_PT they are O_RDONLY but for ja_JP they are O_RDONLY|O_CLOEXEC . So the calls to openat() origin from different locations.