/ oven-sh / bun
Key Takeaways
- Kimi K3 leads with 26 of 48 tasks resolved, followed by Claude Opus 4.8 and Claude Fable 5 at 25.
- GPT-5.6 Sol resolves 24 tasks in 514 seconds, roughly twice as fast as the leading models.
- Kimi K3 and GPT-5.6 Sol reach 27 of 48 together; Sol and Claude Opus 4.7 each contribute a task no other model solves.
Cost / Test vs. Accuracy
GPT-5.6 Luna resolves 23 of 48 tasks for $0.54 per test, offering a lower-cost alternative to the tightly grouped leaders.
Latency vs. Accuracy
GPT-5.6 Sol provides the strongest speed-to-accuracy balance at 24 of 48 tasks in 514 seconds; faster models give up at least three tasks.
Average token use / test
Tasks with failures
| Model | e830f1a | c4d6713 | bfd8e11 | f6f1523 | 2acc042 | 3219883 | d23d693 | 4a53c6b | dd88b5c | 73b6c14 | 8cac538 | be77b65 | f5b73a0 | f64a684 | fdfb820 | a5f045b | f6f22f1 | cdf7f44 | bad2927 | 9f5cc6b | c71f98b | b5f2d0a | 43ee038 | 4cf2d50 | 2908cb3 | 8c170f6 | 8c3c347 | 99fc2f8 | 2e2230a | 8d2a351 | 4bbe075 | 1fbfe1d | 0ecd508 | 5d350cc | 9657f37 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Kimi K3 | |||||||||||||||||||||||||||||||||||
| Claude Opus 4.8 | |||||||||||||||||||||||||||||||||||
| Claude Fable 5 | |||||||||||||||||||||||||||||||||||
| GPT-5.6 Sol | |||||||||||||||||||||||||||||||||||
| Claude Opus 4.7 | |||||||||||||||||||||||||||||||||||
| GPT-5.6 Luna | |||||||||||||||||||||||||||||||||||
| Grok 4.5 | |||||||||||||||||||||||||||||||||||
| Gemini 3.1 Pro Preview (02/26) | |||||||||||||||||||||||||||||||||||
| GPT 5.5 | |||||||||||||||||||||||||||||||||||
| Muse Spark 1.1 | |||||||||||||||||||||||||||||||||||
| Claude Sonnet 5 | |||||||||||||||||||||||||||||||||||
| GPT-5.6 Terra | |||||||||||||||||||||||||||||||||||
| GLM 5.2 | |||||||||||||||||||||||||||||||||||
| Gemini 3.5 Flash | |||||||||||||||||||||||||||||||||||
| Claude Haiku 4.5 (Nonthinking) |
Task detail
e830f1af466bIssue statement
The HTTP request-line parser can mistake complete malformed input for a fragmented request and wait indefinitely for more bytes. Complete request lines must be rejected as soon as a CR/LF terminator proves they cannot become valid: this includes a method token not followed by a space, a one-character invalid method, a short non-origin-form target, and an origin-form target with no HTTP version. At the same time, genuinely partial request-line input must remain eligible for continuation rather than being rejected merely because it is short.
Hidden tests
diff --git a/test/valsmith/http_parser_request_line_test.cpp b/test/valsmith/http_parser_request_line_test.cppnew file mode 100644index 00000000..6fa929c4--- /dev/null+++ b/test/valsmith/http_parser_request_line_test.cpp@@ -0,0 +1,72 @@+#include <algorithm>+#include <chrono>+#include <climits>+#include <cstring>+#include <map>+#include <span>+#include <string>+#include <string_view>+#include <utility>++#define private public+#include "packages/bun-uws/src/HttpParser.h"+#undef private++#include <fstream>+#include <iostream>+#include <vector>++extern "C" size_t BUN_DEFAULT_MAX_HTTP_HEADER_SIZE = 16384;+extern "C" int16_t Bun__HTTPMethod__from(const char *, size_t len) { return len == 1 ? -1 : 0; }++struct Case {+ const char *name;+ std::string input;+ uWS::HTTPHeaderParserError expected;+};++static uWS::ConsumeRequestLineResult parse(std::string input) {+ const size_t input_size = input.size();+ input.resize(input_size + uWS::MINIMUM_HTTP_POST_PADDING, '\r');+ uWS::HttpRequest::Header header{};+ return uWS::HttpParser::consumeRequestLine(input.data(), input.data() + input_size,+ header, true, 16384);+}++static std::string xml_escape(std::string value) {+ size_t pos = 0;+ while ((pos = value.find('&', pos)) != std::string::npos) value.replace(pos, 1, "&");+ pos = 0;+ while ((pos = value.find('\"', pos)) != std::string::npos) value.replace(pos, 1, """);+ return value;+}++int main(int argc, char **argv) {+ const char *report = argc > 1 ? argv[1] : "valsmith-junit.xml";+ const std::vector<Case> cases = {+ {"complete method followed by CRLF is rejected", "GET\r\n\r\n", uWS::HTTP_HEADER_PARSER_ERROR_INVALID_METHOD},+ {"single-character method with a complete line is rejected", "G / HTTP/1.1\r\nHost: localhost\r\n\r\n", uWS::HTTP_HEADER_PARSER_ERROR_INVALID_METHOD},+ {"short non-origin target ending at CRLF is rejected", "POST x\r\n\r\n", uWS::HTTP_HEADER_PARSER_ERROR_INVALID_REQUEST},+ {"origin target without an HTTP version is rejected", "GET /\r\nHost: localhost\r\n\r\n", uWS::HTTP_HEADER_PARSER_ERROR_INVALID_HTTP_VERSION},+ };++ std::ofstream out(report);+ out << "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuite name=\"HttpParser request line\" tests=\""+ << cases.size() << "\">\n";+ int failures = 0;+ for (const auto &test : cases) {+ auto result = parse(test.input);+ bool passed = result.headerParserError == test.expected;+ out << " <testcase classname=\"HttpParser request line\" name=\"" << xml_escape(test.name) << "\">";+ if (!passed) {+ failures++;+ out << "<failure message=\"expected parser error " << int(test.expected)+ << ", got " << int(result.headerParserError) << "\"/>";+ std::cerr << test.name << ": expected parser error " << int(test.expected)+ << ", got " << int(result.headerParserError) << "\n";+ }+ out << "</testcase>\n";+ }+ out << "</testsuite>\n";+ return failures == 0 ? 0 : 1;+}
Head-to-head
Accuracy
54.17%Δ 2.08%
52.08%
Cost / test
$2.26Δ $1.86
$4.11
Latency
1005s
957sΔ 48s
Cost distribution
Latency distribution
Input token distribution
Output token distribution
Task outcomes
48 tasks