# Compatibility Matrix

> Elide's measured pass rates on test262, CPython, javac jtreg, WinterTC and Node-API test suites for the current release, and how to reproduce them.

This page shows Elide's pass rates on upstream suites: Test262, the CPython core tests, javac jtreg, WinterTC and
Node-API. Each rate is one Elide build against one suite revision, from the
[`elide-dev/testsuite`](https://github.com/elide-dev/testsuite) report
[`reports/index.json`](https://raw.githubusercontent.com/elide-dev/testsuite/main/reports/index.json).

## Pass Rate Snapshot

<div class="not-prose my-8 space-y-4">
  <div>
    <div class="mb-1 flex items-center justify-between gap-4 text-sm font-medium">
      <span>JavaScript - test262</span>
      <span>99.9%</span>
    </div>
    <div class="h-3 overflow-hidden rounded-full bg-muted">
      <div class="h-full rounded-full bg-primary" style="width: 99.9%;"></div>
    </div>
  </div>
  <div>
    <div class="mb-1 flex items-center justify-between gap-4 text-sm font-medium">
      <span>Java - javac-jtreg</span>
      <span>99.9%</span>
    </div>
    <div class="h-3 overflow-hidden rounded-full bg-muted">
      <div class="h-full rounded-full bg-primary" style="width: 99.9%;"></div>
    </div>
  </div>
  <div>
    <div class="mb-1 flex items-center justify-between gap-4 text-sm font-medium">
      <span>WinterTC - wpt-wintertc</span>
      <span>95.9%</span>
    </div>
    <div class="h-3 overflow-hidden rounded-full bg-muted">
      <div class="h-full rounded-full bg-primary" style="width: 95.9%;"></div>
    </div>
  </div>
  <div>
    <div class="mb-1 flex items-center justify-between gap-4 text-sm font-medium">
      <span>Python - cpython-core</span>
      <span>93.8%</span>
    </div>
    <div class="h-3 overflow-hidden rounded-full bg-muted">
      <div class="h-full rounded-full bg-primary" style="width: 93.8%;"></div>
    </div>
  </div>
  <div>
    <div class="mb-1 flex items-center justify-between gap-4 text-sm font-medium">
      <span>Node-API - node-api</span>
      <span>86.8%</span>
    </div>
    <div class="h-3 overflow-hidden rounded-full bg-muted">
      <div class="h-full rounded-full bg-primary" style="width: 86.8%;"></div>
    </div>
  </div>
</div>

## Headline Suites

| Area | Suite | Elide build | Upstream revision | Result | Regressions |
| --- | --- | --- | --- | --- | --- |
| JavaScript | test262 | `1.6.0+6e718933f` | `de8e621c` | 99.9% passed: 96,235 pass, 4 fail, 90 error, 7,032 skipped | 0 |
| Python | cpython-core | `1.6.0+6e718933f` | `fd17997c` | 93.8% passed: 31,404 pass, 1,259 fail, 831 error, 1,453 skipped | 0 |
| Java | javac-jtreg | `1.6.0+6e718933f` | `6c48f4ed` | 99.9% passed: 3,925 pass, 1 fail, 2 error, 6 skipped | 0 |

Pass rate is `pass / (pass + fail + error)`; skipped tests are excluded.

## Tracked Suites

These suites also run on every release.

| Area | Suite | Elide build | Upstream revision | Result | Regressions |
| --- | --- | --- | --- | --- | --- |
| WinterTC | wpt-wintertc | `1.6.0+6e718933f` | `1eb456f6` | 95.9% passed: 24,065 pass, 1,018 fail, 5 error, 75 skipped | 0 |
| Node-API | node-api | `1.6.0+6e718933f` | `ed33ae74` | 86.8% passed: 1,194 pass, 176 fail, 5 error, 83 skipped | 0 |

## What the Rates Leave Out

<!-- compat-reasons:start -->
Excluded tests are out of scope for any runtime except the one a suite was written for; skipped tests are in
scope but not run. Neither counts toward a pass rate.

- **Out of scope**: tests of another runtime's internals, other platforms or browser-only features.
- **Not supported**: features Elide does not implement yet, or tests that hang.
- **Test environment**: needs something the machine running the suite does not provide.

### test262

0 excluded, 7,032 skipped.

| Reason | Tests | Status |
| --- | --: | --- |
| **Temporal with non-ISO calendars**. Date arithmetic in calendars such as Ethiopian and Islamic is incomplete. | 4,058 | Not supported |
| **Staging tests**. Tests for proposals not yet part of the standard. | 2,794 | Out of scope |
| **Unshipped proposals**. Legacy RegExp accessors and text, bytes and source-phase imports. | 72 | Not supported |
| **`document.all` emulation**. Needs a browser host object. | 70 | Out of scope |
| **Proper tail calls**. Among major engines only Safari's implements them. | 34 | Not supported |
| **Non-blocking agents**. Checks a throw that applies only where `Atomics.wait` cannot block; Elide can block, as Node does. | 4 | Out of scope |

### cpython-core

2,970 excluded, 1,453 skipped.

| Reason | Tests | Status |
| --- | --: | --- |
| **CPython C API and C accelerators**. Tests of CPython's C interfaces and of C versions of modules Elide ships in Python. | 1,183 | Out of scope |
| **CPython implementation details**. Bytecode, the peephole optimizer, opcode tables and other details the Python docs leave to each implementation. | 902 | Out of scope |
| **Other platforms**. Windows, macOS, BSD, mobile and 32-bit tests. | 799 | Out of scope |
| **Socket ancillary data**. `socket.sendmsg` and `socket.recvmsg`. | 346 | Not supported |
| **Missing os, signal and user-database functions**. Functions such as `os.sendfile`, `os.mkfifo`, `os.timerfd_create`, `signal.pthread_sigmask` and the `grp` and `pwd` modules. | 228 | Not supported |
| **Test machine setup**. Third-party packages, Tk, large memory, terminals, locales, root access or interpreter flags the harness does not provide. | 176 | Test environment |
| **poll and epoll selectors**. `selectors.PollSelector` and `selectors.EpollSelector`. | 151 | Not supported |
| **Specialised socket families**. Bluetooth, CAN, RDS, VSOCK and other kernel socket families the test machine does not load. | 109 | Test environment |
| **Writing bytecode files**. Elide does not write `.pyc` files. | 97 | Not supported |
| **Process forking**. `os.fork`, the fork and forkserver start methods and `os.spawn*`. | 95 | Not supported |
| **Hangs and time limits**. Tests that hang, or run past the per-test limit, under Elide. | 70 | Not supported |
| **Reference-counting timing**. Asserts that an object is freed the moment its last reference goes; Elide uses a tracing garbage collector. | 39 | Out of scope |
| **CPython build options and internals**. Debug, statistics and DTrace builds, reference counts and interpreter internals. | 35 | Out of scope |
| **Other**. 2 tests skipped themselves without recording why; the rest give 121 other reasons. | 193 | Various |

### javac-jtreg

0 excluded, 6 skipped.

| Reason | Tests | Status |
| --- | --: | --- |
| **Ignored upstream**. Tests the JDK project itself marks as ignored. | 4 | Out of scope |
| **JVM launcher flags**. `-J` flags configure a JVM launcher; Elide's compiler is a native binary. | 1 | Out of scope |
| **Custom platform providers**. A `--release` PlatformProvider loaded through service discovery. | 1 | Not supported |

### wpt-wintertc

3,640 excluded, 75 skipped.

| Reason | Tests | Status |
| --- | --: | --- |
| **Browser-only features**. Needs a DOM: documents, frames, navigation, media, `<a>` elements or `XMLHttpRequest`. | 3,438 | Out of scope |
| **CORS enforcement**. Browser security policy (preflight, CORS and CORP checks); Node, Deno and other server runtimes do not enforce it. | 202 | Out of scope |
| **TLS and HTTP/2 test servers**. The test server runs without TLS or HTTP/2. | 75 | Test environment |

### node-api

105 excluded, 83 skipped.

| Reason | Tests | Status |
| --- | --: | --- |
| **Node internals**. Tests of Node's private modules and C++ bindings (`--expose-internals`, `internal/test/binding`). | 105 | Out of scope |
| **`node:cluster`**. Elide does not implement `node:cluster`. | 2 | Not supported |
| **Hangs**. Tests that rely on interrupting a busy child process. | 2 | Not supported |
| **Other**. 68 tests skipped themselves without recording why; the rest give 10 other reasons. | 79 | Various |
<!-- compat-reasons:end -->

## Reproduce The Results

To run the suites yourself, follow [`elide-dev/testsuite`](https://github.com/elide-dev/testsuite). The report
behind this page:

```bash
curl -fsSL https://raw.githubusercontent.com/elide-dev/testsuite/main/reports/index.json
```

For each suite it records the Elide build, the upstream revision, and pass, fail, error, skipped and regression
counts.

## Related Docs

- [JavaScript and TypeScript with Elide](https://elide.help/docs/web)
- [Python with Elide](https://elide.help/docs/python)
- [Java and Kotlin with Elide](https://elide.help/docs/jvm)
