Skip to content
DT Nova

AI in software development: what it does well, the risks and how to use it safely

AI in software development has gone from experiment to everyday tool in two years. Assistants suggest code, write tests, explain legacy systems and review changes; agents carry out whole tasks under supervision. Used well, AI makes good teams faster. Used carelessly, it produces more code that nobody fully understands. This guide covers how widespread AI in software development is, what it does well, where the risks are for quality, security and confidential data, what the EU AI Act means, and the working rules we apply.

Figures are from the AI section of Stack Overflow's 2025 annual survey of people who write code; legal texts are Regulation (EU) 2024/1689 (AI Act) as amended by Regulation (EU) 2026/1744, Regulation (EU) 2024/2847 (Cyber Resilience Act) and the GDPR, all read on 12 November 2026.

Two meanings of AI in software development

The phrase is used for two different things. The first is using AI as a tool while building any software: coding assistants, test generation, code review. The second is building software that contains AI: a chatbot, a document reader, a forecasting model. This guide is about the first. For the second, see our AI consulting and how to make an AI program guide.

How widespread it is

In Stack Overflow's 2025 survey, 84% of respondents said they are using or planning to use AI tools in their development process, up from 76% the year before, and 51% of professional respondents use AI tools daily. Trust has not kept pace: more respondents actively distrust the accuracy of AI tools (46%) than trust it (33%), and only about 3% highly trust the output. The most experienced respondents were the most cautious. That gap between use and trust is the right instinct: AI output is a draft to be checked, not a result.

What AI tools do well today

  • Boilerplate and repetitive code. Data classes, form validation, API clients, configuration files, database migrations.
  • Tests. Generating unit tests for existing functions, including edge cases a tired engineer would skip.
  • Understanding existing code. Explaining what an old module does, in plain language, before anyone changes it. Especially valuable when taking over a system from another supplier.
  • Translation between languages and frameworks. Moving code from an old framework version to a new one, or rewriting a script in another language.
  • First drafts of documentation. Function comments, README files, API descriptions, release notes.
  • Code review support. Flagging likely bugs, missing error handling and inconsistent naming before a human reviewer looks.
  • Queries and data work. SQL queries, data transformations, regular expressions.
  • Agent tasks. Fixing a failing test, applying a change across many files, upgrading a dependency, each checked by an engineer before it is merged.

Where the limits and risks are

  • Plausible but wrong. Generated code often looks right and compiles, while handling an edge case incorrectly. Only tests and review catch it.
  • Insecure patterns. Missing input validation, weak access checks, secrets in code, outdated cryptography. Broken access control is the first item in the OWASP Top 10:2025; AI does not make it less likely on its own.
  • Invented or risky dependencies. Suggestions can reference packages that do not exist, which attackers can then register, or packages with known vulnerabilities.
  • Architecture drift. Each suggestion solves the local problem; nobody designed the whole. Without clear structure, a codebase fills with near-duplicates. See our software architecture guide.
  • More code, same review capacity. If code is produced faster than it can be reviewed, quality falls and technical debt grows.
  • Skills. Juniors who only accept suggestions learn less; teams need deliberate practice in reading and reasoning about code.

Confidential data and trade secrets

Code, database extracts and error logs pasted into an AI tool leave your environment. Check, for each tool, where prompts are processed, whether they are stored, and whether they may be used to train models; business plans usually offer contractual commitments that free consumer tiers do not. Personal data needs extra care: if test data or logs contain names, emails or health information, sending them to an AI service is processing under the GDPR, with a processor that needs a contract under Article 28 and, if it is outside the EU, a valid transfer basis. The simplest rule: no real personal data in prompts, and synthetic test data by default.

What the AI Act means for AI in development

The EU AI Act entered into force on 1 August 2024 and became generally applicable on 2 August 2026. After the 2026 amendments, the rules for high-risk use cases in the areas listed in Annex III apply from 2 December 2027, and those for high-risk AI embedded in products covered by the EU harmonisation legislation listed in the Act's annexes from 2 August 2028. Using an AI assistant to write code is not itself a high-risk use.

One obligation reaches every company that uses AI tools at work. Article 4, as replaced by Regulation (EU) 2026/1744, requires providers and deployers of AI systems to take measures to support the development of AI literacy of their staff and others operating AI systems on their behalf, taking into account their knowledge, experience and the context of use; it does not require any specific level of literacy for any individual. For a software team that means training on what the tools do well, where they fail, and the team's rules for using them. Our AI governance guide covers the wider framework.

The product you build is still your responsibility

Regulation does not care how code was written. Under the Cyber Resilience Act, which applies from 11 December 2027 with reporting obligations from 11 September 2026, manufacturers of products with digital elements must meet its essential cybersecurity requirements and, under Article 13(5), exercise due diligence when integrating components sourced from third parties so that those components do not compromise the product's cybersecurity. A component suggested by an AI tool is a third-party component like any other. The GDPR's data protection by design applies the same way to generated and handwritten code.

Working rules for AI in software projects

  1. Every change is reviewed by an engineer who understands it, whoever or whatever wrote it.
  2. Tests are mandatory for generated code, and tests generated by AI are reviewed as carefully as the code they test.
  3. Security scanning and dependency checks run automatically on every change.
  4. Approved tools only, on business terms, with training on customer code switched off.
  5. No secrets and no real personal data in prompts.
  6. Architecture is designed by people and written down, so AI suggestions fit a structure rather than invent one.
  7. Record which tools are used in the project, so clients and auditors can see it.

Questions to ask a software supplier about AI

  • Which AI tools do your engineers use, on what terms, and is our code used for training?
  • How is generated code reviewed and tested before it reaches our repository?
  • How do you check licences and dependencies of suggested code?
  • Does the contract state that we own all delivered code, however it was produced?
  • How do you keep our personal data and secrets out of AI services?

How we use AI

We use AI tools daily in our own software development work, under the rules above: approved tools on business terms, review and tests on every change, no client personal data in prompts. The result is faster delivery of the routine parts and more time for the parts that need judgement: design, edge cases, security and talking to users. All code, however written, is delivered to a repository in your company's account, and you own it.

Questions people ask

How is AI used in software development?

Mostly as an assistant: suggesting code while typing, generating tests, explaining unfamiliar code, drafting documentation, reviewing changes, writing database queries and converting code between languages. Increasingly also as agents that carry out a multi-step task, such as fixing a failing test, under the supervision of an engineer.

Will AI replace software engineers?

Not in the way the question suggests. AI writes code faster, but deciding what to build, designing a system that will last, judging whether generated code is correct and secure, and taking responsibility for it remain human work. The work shifts from typing code to specifying, reviewing and testing it.

Is AI-generated code safe to use in production?

It can be, if it goes through the same controls as any other code: review by a qualified engineer, automated tests, security scanning and dependency checks. Generated code can contain subtle bugs, outdated practices, insecure patterns or references to packages that do not exist, so it should never skip review.

Does the EU AI Act apply to AI coding tools?

Using an AI assistant to write code is not a high-risk use under the AI Act. Providers and deployers of AI systems do have to take measures to support the AI literacy of their staff under Article 4 as amended in 2026. The software you build with AI help is regulated by what it does, not by how it was written.

Who owns code written with AI tools?

Your contract with the supplier should say that all code delivered, however it was produced, belongs to you, and that the supplier has checked it for licence problems. Check the terms of the AI tools used as well, especially whether your code may be used to train their models.

Sources