Public

cynepiaadmin/cocoindex

Updated: 8/18/2026

Languages

Rust68.8%Python30.9%Shell0.2%Handlebars<0.1%
1 Models30 Tasks

Data transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if you like it!

Add models

Harness

1

Anthropic
Claude Opus 5
Mini-SWE-agent
19 / 30

$3.51

10m55s

Key Takeaways

  • This single 30-task run provides a directional result for Claude Opus 5 with Mini-SWE-agent.
  • Claude Opus 5 with Mini-SWE-agent resolved 63.33% of the supplied tasks.

Cost Analysis

Cost / Test vs. Accuracy
ACCURACYCOST

Average Token Use / Test

Token Usage
InputOutputReasoningCache readCache write

No token usage data available.

Cost is the clearest tradeoff in this comparison. Claude Opus 5 leads at 63.33% for $3.51 per test. No other model in this comparison is cheaper.

Latency Analysis

Latency vs. Accuracy
ACCURACYLATENCY

Average Response Time / Test

Response Time
Claude Opus 5
10m 55s

Claude Opus 5 is both the most accurate and fastest model in this comparison at 63.33% and 10m 55s.

Tasks with failures

Models
Claude Opus 5

Task detail

5308365

Issue statement

LocalFile source listings should be resilient to per-entry filesystem inspection failures. If obtaining an entry's file type or metadata fails (for example, because a symlink cannot be resolved), log a warning, skip that entry, and continue listing other accessible files instead of aborting the entire listing. This applies wherever listing needs metadata, including symlink classification, maximum-size filtering, and ordinal collection.

View Hidden Tests
diff --git a/rust/cocoindex/src/ops/sources/local_file.rs b/rust/cocoindex/src/ops/sources/local_file.rsindex ba21c05..a597147 100644--- a/rust/cocoindex/src/ops/sources/local_file.rs+++ b/rust/cocoindex/src/ops/sources/local_file.rs@@ -296,3 +296,38 @@ impl SourceFactoryBase for Factory {         }))     } }++#[cfg(all(test, unix))]+mod tests {+    use super::*;++    #[tokio::test]+    async fn list_skips_entries_whose_metadata_cannot_be_read() {+        let root = std::env::temp_dir().join(format!(+            "cocoindex-local-file-test-{}",+            uuid::Uuid::new_v4()+        ));+        std::fs::create_dir(&root).unwrap();+        std::fs::write(root.join("healthy.txt"), "healthy").unwrap();+        std::os::unix::fs::symlink("loop", root.join("loop")).unwrap();++        let executor = Executor {+            root_path: root.clone(),+            binary: false,+            pattern_matcher: PatternMatcher::new(None, None).unwrap(),+            max_file_size: None,+            watch_changes: false,+        };++        let rows: Vec<Vec<PartialSourceRow>> = executor+            .list(&SourceExecutorReadOptions::default())+            .await+            .unwrap()+            .try_collect()+            .await+            .expect("an unreadable directory entry should be skipped");++        assert_eq!(rows.len(), 1, "the healthy file should still be listed");+        std::fs::remove_dir_all(root).unwrap();+    }+}diff --git a/.config/nextest.toml b/.config/nextest.tomlnew file mode 100644index 0000000..4cf745a--- /dev/null+++ b/.config/nextest.toml@@ -0,0 +1,2 @@+[profile.default.junit]+path = "junit.xml"