AI system security
Before you connect AI to your business data
Which documents can the system read? Who can it show them to? What data reaches external services? There is a fourth question too: what happens if someone tries to make it do something it should not, perhaps through instructions hidden in a document? We put the answers in writing and test the controls.
What securing an AI system means
It means defining and checking what data an AI application can read, who it can show it to, which actions it can take and what it can send to external services. Using AI to defend a company against cyberattacks is a specialist cybersecurity service. We do not provide that service.
The problems to solve
The system can read more than its users
Nobody knows what leaves the company
A document can contain hidden instructions
What we do
Testing each user's access permissions
We configure the system to answer using only data the person asking is allowed to access. We test the controls with restricted accounts, requesting information that must not appear in their answers.
Documenting what data can leave
We document which data may be sent to external services, which services receive it, in which countries, and how long it is retained. We also specify which data must stay within your company.
Limiting what the system can do
We separate read permissions and credentials from those used to make changes. We require human approval for operations involving prices, payments or personal data.
Testing for misuse before launch
We test hidden instructions in documents, requests beyond the agreed limits and attempts to access other people's data. We use the results to correct the controls before launch and identify what still needs testing.
What we do not do
We do not provide managed monitoring or incident response
Watching your network around the clock and responding to attacks requires a specialist service with dedicated staff, tools and shift coverage. We do not provide that service.
We do not replace your security provider
If you already have a security provider, we work with them. Our decisions about permissions and data must follow your company's security requirements.
We do not declare a system “secure”
We state what we tested, how we tested it and what remains uncovered. A security claim that leaves out what has not been tested is meaningless.
The work your team still needs to do
AI system security needs ongoing maintenance. People, permissions, documents and models change, so even controls that have passed testing need to be reviewed. Someone must keep track of those changes and check that the controls still work.
Keeping permissions current
Reviewing alerts and acting on them
Repeating the tests after every change
What to check and measure
Which document stores can the system access today, and whose permissions does it use?
What data reaches external services, in which countries is it processed, and how long is it retained?
Of twenty attempts to retrieve data without permission, how many succeed?
How many actions can the system complete without human approval?
Who receives alerts about unusual requests, and how long does it take them to review them?
Facts and sources
The General Data Protection Regulation requires technical and organisational measures to ensure a level of security appropriate to the risk (Article 32). It does not prescribe a particular technology: decisions must be proportionate and backed by evidence. That applies to an assistant's permissions and to data sent outside the company.
Among Italian businesses that considered adopting AI but did not do so, 43.2% cite concerns about privacy and data protection as a barrier. Addressing those concerns takes verifiable answers, not reassurance.
Questions we often hear
Will our documents be used to train AI models?
We check that the selected providers' terms exclude using your documents to train models, and we include that restriction in the contract. This needs checking for each AI service; it cannot be assumed.
Where is the data processed?
The location depends on the data involved, the model provider, the regions it operates in and your company's obligations. Before the project starts, we agree with you where the data will be processed and record this in the contract, along with retention periods and which data must never be sent to external services. A project handling customers' personal data has different constraints from one that only searches technical documentation.
What is a hidden instruction in a document?
It is an instruction aimed at the system inside content it is meant to read, such as an email or a PDF you received. If the system treats it as your command, it may disclose data or take unauthorised actions. Action limits and misuse tests help reduce that risk.
Do we need a data protection impact assessment?
It depends on the data and how it will be used; not every project needs one. We assess the need at the start so that, where required, it is part of the work before launch.
Do you handle cyberattacks as well?
No. We do not provide managed monitoring or incident response. Those services need a specialist provider with dedicated staff, tools and shift coverage. We work with your security provider on the AI applications.
How do we know the permissions really work?
We use restricted accounts to request data they should not be allowed to access, and show you the results. A permissions table alone does not show that the controls work.
Is it better to keep everything in-house?
Not always. Running a model on your own premises makes your company responsible for maintenance and updates. The choice depends on the data you handle and who can take on that work.
Let's start with the documents that must stay in-house
Tell us which data must stay within your company and who is allowed to access it. That gives us the basis for defining what the system can do, what access it needs and what to rule out from the start.