ladydascalie/core
Languages
A collection of packages for building a Go microservice on the LUSH platform
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 23 / 30 | $0.01 | $0.44/$1.32 | 6m13s | ||
2 | Mini-SWE-agent | 21 / 30 | $0.09 | $1.4/$4.4 | 2m07s | ||
3 | Mini-SWE-agent | 20 / 30 | $0.04 | $0.4/$1.6 | 83.26s | ||
4 | Mini-SWE-agent | 20 / 30 | $1.07 | $1.5/$7.5 | 3m50s |
Key Takeaways
- GLM 5.2 (Fireworks) with Mini-SWE-agent resolves 21 of 30 tasks at 70%, while Qwen3.7 Plus and Gemini 3.6 Flash each resolve 20.
- Deepseek V4 Flash 0731 with Mini-SWE-agent has the lowest cost per test at $0.01; Qwen3.7 Plus with Mini-SWE-agent is fastest at 83.26 seconds.
Model Comparison
Accuracy
76.67%
DeepSeek V4 Flash 0731
70.00%
GLM 5.2
Task outcomes
30 tasks
Cost / test
$0.01
DeepSeek V4 Flash 0731
$0.09
GLM 5.2
Cost distribution
Latency
6m 13s
DeepSeek V4 Flash 0731
2m 7s
GLM 5.2
Latency distribution
Cost Analysis
Average Token Use / Test
Cost is the clearest tradeoff in this comparison. DeepSeek V4 Flash 0731 leads at 76.67% for $0.01 per test. No other model in this comparison is cheaper.
Latency Analysis
Average Response Time / Test
Latency separates several models with similarly strong scores. DeepSeek V4 Flash 0731 leads at 76.67%, while Qwen 3.7 Plus is fastest at 1m 23s with 66.67% accuracy.
Tasks with failures
| Models | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V4 Flash 0731 | |||||||||||||||
| GLM 5.2 | |||||||||||||||
| Qwen 3.7 Plus | |||||||||||||||
| Gemini 3.6 Flash |
Task detail
b15524aIssue statement
The gRPC pagination middleware currently writes diagnostic messages through the standard logger whenever pagination metadata is missing or malformed. This is noisy for applications that do not want middleware diagnostics. Add a package-level, exported debug flag for paginationmw that defaults to disabled. When disabled, InterceptServerRequest must not log missing or malformed pagination metadata; when enabled, it should retain the existing diagnostic logging. Parsing behavior and returned errors must remain unchanged regardless of the flag.
View Hidden Tests
diff --git a/middleware/paginationmw/debug_logging_test.go b/middleware/paginationmw/debug_logging_test.gonew file mode 100644index 0000000..6ed3fd1--- /dev/null+++ b/middleware/paginationmw/debug_logging_test.go@@ -0,0 +1,59 @@+package paginationmw_test++import (+ "bytes"+ "context"+ "io/ioutil"+ "log"+ "strings"+ "testing"++ "github.com/LUSHDigital/core/middleware/paginationmw"+ "google.golang.org/grpc/metadata"+)++func TestPaginationLoggingIsOptIn(t *testing.T) {+ originalWriter := log.Writer()+ originalFlags := log.Flags()+ originalPrefix := log.Prefix()+ originalDebug := paginationmw.Debug+ defer func() {+ log.SetOutput(originalWriter)+ log.SetFlags(originalFlags)+ log.SetPrefix(originalPrefix)+ paginationmw.Debug = originalDebug+ }()++ log.SetFlags(0)+ log.SetPrefix("")++ t.Run("disabled by default", func(t *testing.T) {+ paginationmw.Debug = false+ var output bytes.Buffer+ log.SetOutput(&output)++ ctx := metadata.NewIncomingContext(context.Background(), metadata.Pairs("per-page", "invalid"))+ if _, err := paginationmw.InterceptServerRequest(ctx); err == nil {+ t.Fatal("expected malformed metadata to return an error")+ }+ if output.Len() != 0 {+ t.Fatalf("debug logging is disabled, but got log output %q", output.String())+ }+ })++ t.Run("enabled", func(t *testing.T) {+ paginationmw.Debug = true+ var output bytes.Buffer+ log.SetOutput(&output)++ ctx := metadata.NewIncomingContext(context.Background(), metadata.Pairs("per-page", "invalid"))+ if _, err := paginationmw.InterceptServerRequest(ctx); err == nil {+ t.Fatal("expected malformed metadata to return an error")+ }+ if !strings.Contains(output.String(), "could not parse") {+ t.Fatalf("expected a parsing diagnostic, got %q", output.String())+ }+ })++ log.SetOutput(ioutil.Discard)+}