Skip to content

feat(databricks): add Databricks integration with 8 tools#3361

Merged
waleedlatif1 merged 4 commits intostagingfrom
waleedlatif1/add-databricks
Feb 27, 2026
Merged

feat(databricks): add Databricks integration with 8 tools#3361
waleedlatif1 merged 4 commits intostagingfrom
waleedlatif1/add-databricks

Conversation

@waleedlatif1
Copy link
Collaborator

Summary

  • Add complete Databricks integration with 8 tools: execute_sql, list_jobs, run_job, get_run, list_runs, cancel_run, get_run_output, list_clusters
  • Auth via Personal Access Token + workspace host URL
  • Structured object outputs with destructured fields for known response shapes
  • mode: 'advanced' for optional fields, wandConfig for JSON params and timestamps
  • All endpoints verified against official Databricks API docs

Type of Change

  • New feature

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Add complete Databricks integration supporting SQL execution, job management,
run monitoring, and cluster listing via Personal Access Token authentication.

Tools: execute_sql, list_jobs, run_job, get_run, list_runs, cancel_run,
get_run_output, list_clusters

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Feb 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Feb 27, 2026 3:01am

Request Review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 27, 2026

Greptile Summary

Added complete Databricks integration with 8 tools covering SQL execution, job management, and cluster operations. Implementation follows established patterns with proper error handling, structured outputs, and comprehensive UI configuration.

Key changes:

  • 8 new tools: execute_sql, list_jobs, run_job, get_run, list_runs, cancel_run, get_run_output, list_clusters
  • Structured object outputs with destructured fields (camelCase) from Databricks API responses (snake_case)
  • Wand AI support for JSON parameter generation (jobParameters, notebookParams) and timestamp conversion
  • Proper credential handling with user-only visibility for apiKey and host
  • Advanced mode for optional parameters (expandTasks, includeHistory, filtering options)
  • Previous review feedback addressed: JSON parsing now throws proper errors, expandTasks UI field added

Issue found:

  • Documentation error: waitTimeout default value documented as "10s" but code uses "50s"

Confidence Score: 4/5

  • Safe to merge after fixing documentation error
  • Implementation is solid with proper error handling, type safety, and follows project patterns. Previous review feedback has been addressed. Only issue is a minor documentation inconsistency that should be corrected before merge.
  • apps/docs/content/docs/en/tools/databricks.mdx - fix waitTimeout default value documentation

Important Files Changed

Filename Overview
apps/sim/tools/databricks/types.ts Clean TypeScript type definitions for all 8 Databricks tools with proper parameter interfaces and response types
apps/sim/tools/databricks/execute_sql.ts Well-implemented SQL execution tool with proper error handling, response transformation, and structured output
apps/sim/tools/databricks/run_job.ts Job triggering tool with proper JSON parsing error handling (addresses previous review feedback)
apps/sim/tools/databricks/list_jobs.ts Jobs listing tool with pagination support and expandTasks parameter properly implemented
apps/sim/blocks/blocks/databricks.ts Comprehensive block configuration with all 8 operations, conditional UI fields, wand AI support for JSON params/timestamps, and expandTasks UI field (addresses previous feedback)
apps/docs/content/docs/en/tools/databricks.mdx Comprehensive documentation for all tools, but contains incorrect default value for waitTimeout (says "10s" instead of "50s")

Sequence Diagram

sequenceDiagram
    participant User
    participant Block as Databricks Block
    participant Tool as Databricks Tool
    participant API as Databricks API
    
    User->>Block: Select operation (e.g., execute_sql)
    User->>Block: Provide credentials (host, apiKey)
    User->>Block: Configure parameters
    Block->>Block: Transform params (strings to numbers/booleans)
    Block->>Tool: Route to appropriate tool (databricks_execute_sql)
    Tool->>Tool: Build request URL & headers
    Tool->>Tool: Construct request body
    Tool->>API: POST/GET request with Bearer token
    API-->>Tool: JSON response
    Tool->>Tool: Transform response (snake_case to camelCase)
    Tool->>Tool: Destructure fields into structured output
    Tool-->>Block: Return typed response
    Block-->>User: Display results with destructured fields
Loading

Last reviewed commit: af865eb

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

18 files reviewed, 4 comments

Edit Code Review Agent Settings | Greptile

…add expandTasks field

- Throw errors on invalid JSON in jobParameters/notebookParams instead of silently defaulting to {}
- Always set boolean params explicitly to prevent string 'false' being truthy
- Add missing expandTasks dropdown UI field for list_jobs operation
- execute_sql: fix wait_timeout default description (50s, not 10s)
- get_run: add queueDuration field, update lifecycle/result state enums
- get_run_output: fix notebook output size (5 MB not 1 MB), add logsTruncated field
- list_runs: add userCancelledOrTimedout to state, fix limit range (1-24), update state enums
- list_jobs: fix name filter description to "exact case-insensitive"
- list_clusters: add PIPELINE_MAINTENANCE to ClusterSource enum
@waleedlatif1
Copy link
Collaborator Author

@greptile

@waleedlatif1
Copy link
Collaborator Author

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@waleedlatif1 waleedlatif1 merged commit c8a0b62 into staging Feb 27, 2026
11 checks passed
@waleedlatif1 waleedlatif1 deleted the waleedlatif1/add-databricks branch February 27, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant