Buffaly Logo
Catalog

Browser, voice, and integrations

Choose the right Buffaly boundary for browser work, local computer use, voice handoffs, data access, and external services.

Quick answer

Integrations are separate operating boundaries. Finding source code, a module folder, or a wiki page does not prove the integration is enabled for your runtime or user.

Verify an installed module entry or deploy artifact.

Verify the skill, action, runtime route, and service binding.

Verify credentials, account identity, and provider access.

Verify user approval, permission scope, and write boundaries.

Native Buffaly capabilities run inside the product boundary. External integrations operate through browser automation, gateway services, provider APIs, database helpers, credentials, and user scopes. Treat those as explicit contracts, not assumptions.

How to choose

Surface Use when Confirm first
Local filesThe work is in source, docs, logs, or local artifacts.Workspace path, file permissions, and whether edits are allowed.
BrowserA visual site, logged-in session, or page workflow is the real interface.Browser module install, route, account, and approval before submit.
ComputerUseThe task requires desktop UI, local apps, or full computer interaction.Runtime availability, action binding, screen context, and safety stop.
SQL ServerYou need structured reads, previews, table handles, or diagnostics.Connection string, secret-key password resolution, and read-only limits.
Typed service, HTTP, or JsonWsA domain API is available and repeatability matters more than a visual path.Owner service, endpoint, auth, schema, retries, and allowed operations.
AWS SSMYou need a controlled interactive session on a configured instance.AWS CLI, profile, region, instance id, timeout, and no-write boundary.
SMS and voiceThe workflow is mediated by a telephony gateway or conversation handoff.Gateway route, draft-versus-send policy, transcript, and protected credentials.
Google WorkspaceWork belongs in Gmail, Drive, Docs, Sheets, Calendar, or Chat.Installed APIs, OAuth account, scopes, and target document or mailbox.
Image generationYou need generated images or edits through a configured provider.Provider binding, prompt safety, output location, and usage policy.

Configuration gates

Installed artifact

The module, skill file, action, page, or provider package must exist in the deployed runtime, not only in a source checkout.

Runtime registration

Routes, web modules, action bindings, service registrations, and prompts must be discoverable by the running app.

Identity and credentials

The correct user, tenant, OAuth grant, API key, database secret, or gateway credential must be present and protected.

Approval and scope

Buffaly needs a clear boundary: inspect, draft, ask before sending, or complete the action after validation.

Manifest note: WebModules.json entries commonly identify pages for Heartbeat, Wiki, SkillManagement, GoogleWorkspace, Browser, ComputerUse, and OpenAIImageGeneration. A manifest entry is necessary for module discovery, but the service, action, credential, and user-scope gates still matter.

Browser module

Use Browser when the website itself is the source of truth: visual state, authenticated pages, forms, screenshots, or workflows that do not expose a better API.

Integration map

Look for Browser artifacts, the manifest page, skill file, BrowserSkill, BrowserAction, BrowserWebModuleService, BrowserWebModuleService#Local, ToRunBrowserModuleTask trigger phrases, and BrowserContextPrompt.

Good requests

"Open the billing portal, find the latest invoice, and stop before downloading." "Check whether the campaign form accepts this payload and report visible validation errors."

Provide

Target URL, account/environment, success criteria, whether to inspect or submit, and any evidence you need captured.

  • Verify the Browser page appears in the runtime module list.
  • Verify the skill/action can route a browser task.
  • Common failures: no module registration, missing session, blocked login, ambiguous submit permission, or visual state that changed mid-task.
  • Prefer an API or typed service when the site is only a wrapper over structured data.

ComputerUse

Use ComputerUse when the task depends on the desktop, installed applications, screen state, or local UI interactions that Browser cannot reach.

Confirm the ComputerUse artifacts, manifest page, skill, service, and action binding before planning a workflow. Ask for bounded actions such as "open the app, inspect the export settings, and stop before saving" or "compare the visible local report against this expected value."

Provide

Application name, visible state, account, file path, and stop conditions.

Verify

Runtime route, service binding, screen access, and user approval for local changes.

Failures

Unavailable desktop session, missing action, hidden modal, wrong window, or unsafe write scope.

SQL Server

Use SQL Server helpers for read-only diagnostics, previews, table handles, and structured investigation when the database is the right boundary.

Connection setup may resolve a password through a secret key before building the connection string. Preview caps and tabular handles keep responses reviewable and avoid dumping large data sets into chat.

Good requests

"Preview the top 50 failed imports by created date." "Show the schema for the customer table and summarize nullable columns."

Provide

Server or connection name, database, table or query intent, expected filters, and whether sensitive fields must be excluded.

Verify and troubleshoot: confirm the helper is read-only, credentials resolve, the connection string points at the intended environment, preview caps are acceptable, and failures are not caused by network, permission, or secret lookup issues.

HTTP and JsonWs

Use a typed service when one owns the domain. Use HTTP or JsonWsHelper when no typed service exists and the target exposes a stable endpoint.

JsonWsHelper is useful for JSON web-service calls, request construction, response parsing, and structured errors. Provide the endpoint, method, headers or auth source, request body, expected schema, idempotency concerns, and allowed side effects.

Common failures: missing credentials, wrong environment, undocumented response shape, rate limits, non-idempotent retries, or using a generic HTTP call where a typed Buffaly service already owns the operation.

AWS SSM

Use SsmInteractiveSessionService for controlled interactive access to a configured AWS instance when that is the approved operations boundary.

Provide

AWS profile, region, instance id, timeout, command intent, and whether the session is inspect-only.

Verify

AWS CLI availability, profile permissions, SSM agent state, network reachability, and no-write boundary before changes.

Common failures include expired AWS identity, wrong region, an instance without SSM registration, timeout, or a request that crosses from observation into mutation without approval.

SMS and voice gateway

SMS and voice workflows are gateway-mediated. Treat drafting, sending, calling, transcription, and follow-up as separate approval steps.

Confirm route and gateway configuration categories, context prompts, protected credentials, sender or caller identity, and whether Buffaly should prepare a draft or send through the gateway.

Good requests

"Draft a reply from this call summary and ask before sending."

Provide

Recipient, message intent, required disclosures, and send approval rule.

Failures

Gateway disabled, missing credentials, blocked number, ambiguous consent, or no transcript context.

Google Workspace

Use Google Workspace when the work belongs in Gmail, Drive, Docs, Sheets, Calendar, or Chat and the configured OAuth account has the right scopes.

Verify the WebModules.json page entry, installed API coverage for the product you need, account identity, OAuth consent, scopes, and whether Buffaly may read, draft, edit, share, or send.

Good requests name the app and object: "Find the latest vendor invoice email and draft a summary," "Create a Sheet tab from this table," or "Check my calendar for open times next Tuesday." Common failures are wrong account, insufficient scopes, expired OAuth, missing Drive access, or unclear write approval.

Image generation

Use image generation when the installed OpenAI image module owns the provider call and output handling.

Confirm the OpenAIImageGeneration manifest page, provider configuration, output destination, prompt safety requirements, and whether the request is for a new image, edit, variation, or asset handoff. Source presence alone does not mean the provider is configured for the current user.

Local development and code surfaces

File system and ripgrep

Use local files and rg for source, docs, logs, and configuration searches inside the allowed workspace.

Process and PowerShell

Use process commands for diagnostics, tests, local scripts, and environment checks when command execution is available.

Visual Studio and code search

Use IDE and search surfaces for navigating projects, symbol references, and solution-level context.

Codex

Use Codex for scoped code edits, reviews, repo-aware implementation, and verification within the requested files.

How to ask Buffaly for integration work

Browser example

Open the customer portal in the staging account, confirm whether invoice 1042 is marked paid, capture the visible evidence, and stop before changing anything.

Service example

Use the configured API to fetch the last ten failed webhook deliveries, summarize status codes, and do not retry deliveries without approval.

Workspace example

In the finance Drive folder, find the latest budget Sheet, create a draft summary tab, and ask before sharing it.

Voice example

From the call transcript, draft a follow-up SMS with the agreed next step and wait for confirmation before sending.

Troubleshooting and verification checklist

  • Confirm the installed artifact exists in the running deployment.
  • Confirm WebModules.json, route, skill, action, service, or provider registration as applicable.
  • Confirm the current user, tenant, account, OAuth grant, API key, database secret, or gateway credential.
  • Confirm the requested action is inside the approved scope: inspect, draft, ask, or complete.
  • Prefer native Buffaly services or typed APIs before browser automation when the domain has a reliable service boundary.
  • For failures, identify whether the problem is discovery, routing, credentials, permission, provider response, rate limit, or changed external state.