improved cache handling

[narengogi]

improved cache handling with methods to use the default cache in different environments (node, workerd)

please merge this after APM changes #1394 (this branch has changes from that branch merged)

[matter-code-review]

Description

Summary By MatterAI

🔄 What Changed

• Added MatterAI provider integration with full config, API, and streaming support.
• Enhanced cache handling in panw-prisma-airs plugin with trace ID propagation and secure fallback behavior.
• Introduced comprehensive unit tests for AIRS guardrail with mocking and behavioral validation.
• Extended Cohere provider with full schema support, streaming logic, and finish reason mapping.
• Added Google Vertex AI tooling support for search, retrieval, and computer use.
• Integrated patch-package for post-install patches and updated dependencies.

🔍 Impact of the Change

• Enables seamless integration with MatterAI models via OpenAI-compatible interface.
• Improves security and observability in AIRS scanning with trace correlation and graceful API key failure.
• Increases test coverage and reliability of guardrail logic with 224 new test lines.
• Enhances multi-tool agent capabilities via Google Vertex AI tool transformations.
• Ensures long-term maintainability with dependency patching via patch-package.

📁 Total Files Changed

File	ChangeLog
package.json	Added hono and patch-package dependencies
intercept.ts	Enhanced AIRS cache handling with trace ID and fallback logic
panw.airs.test.ts	Added 224 lines of unit tests for AIRS handler
globals.ts	Added MATTERAI provider constant
chatComplete.ts	Refactored Cohere chat with full streaming and schema support
embed.ts	Enhanced Bedrock embed response parsing
getBatchOutput.ts	Added error handling and Amazon Titan model support
api.ts	Updated Cohere API endpoints and base URL
types.ts	Added Cohere stop reasons and streaming types
utils.ts	Added Google tool transformers and state handling
index.ts	Integrated MatterAI provider config
matterai/api.ts	Created MatterAI API config
matterai/chatComplete.ts	Implemented MatterAI chat config and stream transform
matterai/index.ts	Bootstrapped MatterAI provider module

🧪 Test Added/Recommended

Added

• Mocked Unit Tests: Full Jest mock of post utility with 10 test cases covering: API key missing, network errors, malformed responses, block/allow verdicts, trace ID propagation.
• Behavioral Testing: Verified x-portkey-trace-id is correctly passed as tr_id in AIRS payload.
• Error Handling: Confirmed no HTTP call occurs when API key is missing/empty.
• Profile Support: Validated profile_name, profile_id, and app_name parameter handling.

Recommended

• Add integration tests with real AIRS endpoint using test credentials.
• Implement end-to-end tracing validation to ensure trace ID consistency across services.
• Add performance benchmarks for cache hit/miss scenarios in fetchAIRS.

🔒Security Vulnerabilities

• Input Validation: No validation on traceId generation from untrusted headers — potential for header injection if used downstream.
• Error Leakage: Missing API key returns detailed error message that could expose internal implementation.
• Crypto Fallback: Insecure Math.random() fallback when crypto.randomUUID is unavailable — risk of predictable IDs.
• Dependency Risk: patch-package added but no lockfile changes visible — ensure patches are audited.

⏳ Estimated code review effort

HIGH (~45 minutes)

Tip

Quality Recommendations

1. Validate and sanitize 'x-portkey-trace-id' header input to prevent injection attacks

2. Replace Math.random() fallback with cryptographically secure alternative for trace ID generation

3. Implement input validation for profile_name, profile_id, and app_name parameters

4. Add rate limiting on AIRS API calls to prevent abuse

5. Use structured logging for AIRS request/response payloads for better observability

6. Add timeout to fetchAIRS call to prevent hanging requests

7. Implement cache layer for AIRS responses to improve performance

8. Move error message about missing API key to debug level to avoid information leakage

♫ Tanka Poem

Cache flows through wires,
Trace IDs dance in requests—
Secure, swift, smart.
New tools rise in Vertex light,
MatterAI joins the stream.

Sequence Diagram

sequenceDiagram
    participant Client
    participant AIRSHandler
    participant Utils
    participant AIRSAPI
    
    Client->>AIRSHandler: handler(ctx, params)
    AIRSHandler->>Utils: getText(ctx, hook)
    Utils-->>AIRSHandler: prompt/response text
    
    alt API Key Missing
        AIRSHandler-->>Client: {verdict: true, error: 'AIRS_API_KEY required', data: null}
    else API Key Present
        AIRSHandler->>AIRSHandler: Extract x-portkey-trace-id
        
        alt Trace ID Missing
            AIRSHandler->>AIRSHandler: Generate fallback ID
        end
        
        AIRSHandler->>AIRSHandler: Build payload with traceId
        AIRSHandler->>AIRSHandler: Add profile/app metadata
        AIRSHandler->>AIRSHandler: fetchAIRS(payload, apiKey)
        AIRSHandler->>AIRSAPI: POST /v1/scan/sync/request
        
        alt Response OK
            AIRSAPI-->>AIRSHandler: {action: 'allow'|'block'}
            AIRSHandler-->>Client: {verdict: action==='allow'}
        else Response Malformed
            AIRSAPI-->>AIRSHandler: Invalid response
            AIRSHandler-->>Client: {verdict: false, error: 'Malformed AIRS response'}
        else Network Error
            AIRSAPI-->>AIRSHandler: Throws error
            AIRSHandler-->>Client: {verdict: false, error: network error}
        end
    end

Loading

[matter-code-review]

✅ Reviewed the changes: Improved cache handling, logging, and Prometheus metrics with pluggable backends. Refactored provider configurations.

[matter-code-review]

✅ Reviewed the changes: Improved cache handling and added MatterAI provider support with updated Cohere and Bedrock transformations.