Self-Hosted AI Privacy: Audit Where Prompts and Files Actually Go

Running a model on your own server can give you more control over data handling. It doesn't automatically mean every prompt, attachment, or generated answer stays there. The surrounding application may call cloud models, web search, embedding services, monitoring tools, or a support platform.

A self-hosted AI privacy audit follows the data through the entire request. The question is practical: what leaves the boundary, for what purpose, and under whose configuration? Answer that before describing the service to users as private or local-only.

Draw the real request path

List the browser, API gateway, inference server, embedding service, document store, tool integrations, and telemetry destinations. Include authentication and file-processing services. A diagram that ends at the model misses much of the application.

For every step, record the data received and any onward request. Does an embedding call receive full paragraphs? Does a search tool send a query containing customer details? Does an error reporter capture the request body? These details are more useful than a broad “hosted on our server” statement.

Use a synthetic document and prompt while testing. Don't use a real confidential file to discover whether the system sends confidential files elsewhere. Give the test data distinctive, harmless markers that can be searched in authorised logs.

Check local and cloud modes explicitly

Some runtimes support both local and remote capabilities. Review the mode actually enabled, not the product category. Ollama's FAQ documents local-only configuration and the effect of disabling its cloud features.

For that runtime, the documented environment setting includes:

OLLAMA_NO_CLOUD=1

Confirm support in your installed version, apply the setting through the approved configuration process, and restart as required. Then inspect the relevant status or logs. A configuration file that looks right is weaker evidence than a tested running configuration.

This setting concerns that runtime's cloud features. It doesn't disable a separate application plugin, an external embedding client, or your own monitoring exporter. Keep those components in the audit rather than treating one switch as a complete privacy guarantee.

Review tools and fallback paths

Ask what happens when the local model is unavailable, too slow, or unable to process a request. A cloud fallback may improve availability while changing the data boundary. Make that behaviour explicit and obtain any required authorisation before sensitive data can take the fallback route.

Review web-search queries, browser actions, and file converters too. An agent can send data outward through a tool even when inference stays local. Restrict tool permissions according to the application purpose and test them with the synthetic markers.

For a sanitised integration manifest, JSON Formatter can make destinations and enabled capabilities easier to review. It doesn't inspect network traffic or prove that the application follows the manifest.

Observe authorised network behaviour

Use your own network monitoring and application instrumentation in a controlled environment to compare intended and observed destinations. Check startup, model download, normal inference, failure paths, and administrative operations. Some connections occur only during maintenance or errors.

Distinguish downloading a model from uploading a prompt. Both are network activity, but they have different privacy implications. Record request purpose and data type without unnecessarily retaining content in the monitoring system.

Test a deliberately unavailable local model and a failed tool call. These failures often expose fallback paths that ordinary successful inference never exercises. Record whether the request stops, retries locally, or contacts another service, then compare that behaviour with the promised boundary.

Don't make a universal “no data leaves” claim from one quiet test. State the environment, version, configuration, and paths examined. If a component cannot be inspected, record the limitation and consult its official documentation or vendor.

Include storage and people in the boundary

Prompts can remain inside your network and still be overexposed. Review administrator access, shared disks, backups, support exports, and retention. A local log containing every conversation may be more accessible than the original application data.

Define who can view raw prompts and how long they are kept. Prefer operational events that describe outcomes without capturing full sensitive payloads. Use synthetic data to check whether error messages or tracing spans unexpectedly include the original document.

Also review where the hosting provider's staff or subprocessors may have access. Self-hosting in a rented environment still involves infrastructure relationships. Confirm contractual and regulatory questions with qualified support rather than inferring them from a server location.

Make privacy checks part of updates

Compare sanitised configuration revisions with JSON Diff. Flag new destinations, newly enabled fallbacks, request-body logging, and changed retention. A routine application upgrade can change privacy behaviour without changing the model.

Keep a test set that exercises each outbound capability and checks expected restrictions. Re-run it after changing the runtime, integrations, or monitoring stack. Assign an owner to investigate unexpected destinations instead of leaving the result in an unreviewed dashboard.

Before serving downloaded weights, also review the separate model licence checklist. Data boundaries, artifact permissions, and application quality need different evidence; none of them proves the others.

Conclusion

Self-hosting creates an opportunity for control, not an automatic privacy promise. Trace inference, tools, fallbacks, telemetry, storage, and access. Test the running configuration with synthetic data, document limitations, and repeat the audit when the surrounding application changes.