Test HHVM on ARM64

This page contains instructions regarding testing HHVM on ARM64 as well a current test results.

Instructions

After building hhvm simply invoke the following command:
./hphp/hhvm/hhvm hphp/test/run all
This will run all tests. If you want to use other test sets (e.g. quick or slow) simply replace 'all' by the desired set name. If you want to write the output into a file, you can get rid of the progress bar by using the flag '-v'.

Status

The lists below contain only failed tests, which can be reproduced on one of the listed arm64 Linux distributions (Ubuntu Xenial, Fedora 25, CentOS7 and OpenSuse LEAP 42.2). That implies, that failed tests, which are observed on a x64 distribution, are not listed here, unless they are failing on at least one arm64 distribution.

Fix pending (patch/PR exists)

UFCS cf
xxxx xx hphp/test/slow/ext_file/empty_filename.php.diff #bug in boost (see PR #7534)
xxxx xx hphp/test/slow/program_functions/config_overrides.php #bug in testcase (see PR #7619)
xxxx xx hphp/test/slow/program_functions/tier_overrides_default.php #bug in testcase (see PR #7619)
xxxx xx hphp/test/slow/program_functions/tier_overrides_tier.php #bug in testcase (see PR #7619)
nxxn xx hphp/test/zend/good/ext/openssl/tests/bug36732.php #MD5 certificate verification disabled (see PR #7651)

U...Ubuntu Xenial (4 fails)
F...Fedora 25 (5 fails)
C...CentOS 7 (5 fails)
S...OpenSuse LEAP 42.2 (4 fails)
c...x64 CentOS 7 (5 fails)
f...x64 Fedora 25 (5 fails)

n...not triggered
x...triggered

Wontfix

The following list contains failing selftests, which are caused by wrong assumptions about the environment (e.g. systems with a specific version of a library/application needs to be installed, or specific configuration options set, etc.). In fact the test cases are wrong, but fixing them might be a pointless overkill. Instead the failing tests are listed here with their behaviour.
UFCS cf
nnnx nn hphp/test/slow/ext_openssl/signature_alg.php #Suse Linux has additional hash functions "MDC2" and "mdc2"
xxxn nx hphp/test/slow/ext_mb/mb_sendmail_null.php #Different behaviour of different MTAs (HHVM executes /usr/sbin/sendmail which might be sendmail, esmtp, postfix wrapper...)
nxnn nx hphp/test/slow/ext_intl/numberformat_other.php #Locale data variation
nxnn nx hphp/test/slow/ext_icu/date_pattern_gen.php #Locale data variation
nnxn xn hphp/test/slow/ext_icu/ext_icu.php #libicu data variation
xxnn nx hphp/test/slow/ext_icu/uspoof.php #libicu data variation
nnnx nn hphp/test/slow/invalid_argument/1381.php #locale data variation
nnxx nn hphp/test/slow/locale/locale-stack-overflow.php #crash of libicu (does not happen with version 55+)
xxxx xx hphp/test/zend/good/ext/date/tests/bug33414-1.php #Bug in Linux locale data and timezone representation variation
xxxx nn hphp/test/zend/good/ext/gettext/tests/gettext_basic.php #Bug in gettext(); see https://bugzilla.redhat.com/show_bug.cgi?id=1418635
nnxn xn hphp/test/zend/good/ext/intl/tests/resourcebundle_null_mandatory_args.php #missing feature in libicu (does not happen with version 52+)
xxnn nx hphp/test/zend/good/ext/intl/tests/spoofchecker_004.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/timezone_getDisplayName_variant2-49+.php #localization data variation
xxnn nx hphp/test/zend/good/ext/intl/tests/timezone_getDisplayName_variant3-49+.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/dateformat_format_parse.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/dateformat_get_set_timezone.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/formatter_get_locale.php #localization data variation
nxnn nx hphp/test/zend/good/ext/intl/tests/formatter_get_set_text_attribute.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/locale_filter_matches2.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/locale_get_display_name2.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/locale_get_display_region2.php #localization data variation
nnxn xn hphp/test/zend/good/ext/intl/tests/locale_lookup.php #localization data variation

U...Ubuntu Xenial (6 fails)
F...Fedora 25 (9 fails)
C...CentOS 7 (14 fail)
S...OpenSuse LEAP 42.2 (5 fail)
c...x64 CentOS 7 (11 fails)
f...x64 Fedora 25 (8 fails)

n...not triggered
x...triggered

Open bugs

UFCS cf
nnnx nn hphp/test/quick/debugger/printThis.php
xxxx xx hphp/test/slow/ext_shmop/ext_shmop.php #hack typechecker crash
xxxx xx hphp/test/slow/ext_watchman/ext_watchman.php #regression (see IS #7649)
xxxx xx hphp/test/slow/ini/recursive_substitutions1.php #hack typechecker crash
xxxx xx hphp/test/slow/ini/recursive_substitutions2.php #hack typechecker crash
xxxx xx hphp/test/slow/ini/recursive_substitutions3.php #hack typechecker crash
xxxx xx hphp/test/slow/namespace/alias/alias.php #hack typechecker crash
xxxx xx hphp/test/zend/good/ext/calendar/tests/easter_date.php #TZ variable is not propagated to libc (HHVM bug #7533)
nnxn xn hphp/test/zend/good/ext/date/tests/DateTimeZone_getTransitions_basic1.php
xxxx xx hphp/test/zend/good/ext/date/tests/bug20382-2.php #timezone representation variation
nnxn xn hphp/test/zend/good/ext/date/tests/bug33414-2.php #timezone representation variation
xxxx xx hphp/test/zend/good/ext/date/tests/bug33415-1.php #timezone representation variation
nnxn xn hphp/test/zend/good/ext/date/tests/date_sunrise_variation9.php
nnxn xn hphp/test/zend/good/ext/date/tests/date_sunset_variation9.php
nnxn xn hphp/test/zend/good/ext/date/tests/getdate_variation7.php
xxxx xx hphp/test/zend/good/ext/date/tests/timezone_location_get.php #Unexpected result 59.91667
xxxx xx hphp/test/zend/good/ext/mbstring/bug28220.php #bug in mb_strwidth() (see IS #7643)
xnnx nn hphp/test/zend/good/ext/mbstring/tests/mb_ereg_replace_variation1.php #regression (see IS #7650)
xnnx nn hphp/test/zend/good/ext/mbstring/tests/mb_ereg_variation3.php #regression (see IS #7650)
xnnn nn hphp/test/zend/good/ext/xsl/tests/bug33853.php #crash of HHVM (due to behaviour change in libxml2)
xxnx nx hphp/test/zend/good/ext/xsl/tests/bug53965.php #libxslt/libxml2 behaviour change (does not allow file:// protocol anymore?)

U...Ubuntu Xenial (15 fails)
F...Fedora 25 (12 fails)
C...CentOS 7 (16 fails)
S...OpenSuse LEAP 42.2 (15 fails)
c...x64 CentOS 7 (16 fails)
f...x64 Fedora 25 (12 fails)

n...not triggered
x...triggered
t...triggered, more testing required

Notes

No fastcgi tests included

Note that fastcgi tests are unstable at the moment (work sometimes). They are currently excluded from the list above. See also IS #7615.

Reasons for other failing tests

Other tests might fail, because:

 *) Some tests assume that all locales are created
    (not default on Debian/Ubuntu/etc.).
    Can usually be fixed with:
    # cp /usr/share/i18n/SUPPORTED /etc/locale.gen
    # locale-gen

 *) HHVM test cases are not multi-user safe.
    Can be circumvented with:
    rm -rf /tmp/lock.txt \
           /tmp/trace* \
           /tmp/perf* \
           /tmp/sess* \
           /tmp/u_sess* \
           /tmp/trace* \
           /tmp/hhvm* \
           /tmp/stacktrace* \
           /tmp/test* \
           /tmp/remote_eval.test

 *) HHVM assumes the test user is in the group wheel.
    Can be achieved with:
    groupadd wheel
    usermod -a -G wheel myuser