A penetration test is paid by the day. Every day the testers spend waiting for an account, guessing how a function is meant to work or being blocked by your own firewall is a day not spent on finding vulnerabilities. Most of that waiting can be removed before the start.
Decide
What is the question? “Is the new payment flow safe to release” and “what can someone on the internet do to us” lead to different tests. Write the question down in one sentence; the scope follows from it.
What is in scope, and what is not? List the applications, the addresses, the environments. List the exclusions too: the payment provider, the systems of the parent company, the legacy service that falls over when it is looked at.
Which environment? A staging environment that matches production is the best choice: the testers can be thorough and nothing real is at stake. If only production exists, agree the hours and the techniques that are off limits.
How much do the testers know? Testing with accounts and documentation finds more per day than testing blind. Testing blind answers one narrow question: what an outsider achieves with no information. Decide which you are paying for.
Prepare
Accounts. One for each role, and in two separate tenants if the product has tenants: access control between customers cannot be tested with one customer. Create them before the start, log in with each once, and check that they have realistic data.
Documentation. API specifications, a diagram of the architecture, a description of the roles and of the main workflows. Imperfect documents are better than none.
Access. If the test environment is behind a VPN or an allow-list, arrange access in advance and test it. Decide whether the web application firewall stays on. If the application is the subject of the test, allow the testers through it; if the firewall is, leave it on and say so.
Data. Populate the test environment with data that resembles the real thing in structure and not in content. Real personal data in a test environment is a finding before the test has begun.
Backups. Confirm that the environment under test can be restored. Testers are careful; backups are for the case in which care was not enough.
Tell
The operations team and the monitoring team, unless the test is meant to examine them. Give them the source addresses of the testers and the dates. Otherwise the first day ends with the testers blocked and an incident opened.
The hosting or cloud provider, where their policy requires it. The large cloud providers do not require notice for testing of your own resources within their published rules; smaller hosting companies often do.
Suppliers whose systems are touched. Their consent is needed in writing.
One contact person, reachable during testing hours, with the authority to make decisions: to extend an account, to restart a service, to stop the test.
Sign
- the non-disclosure agreement, before any of the above is handed over;
- the contract;
- the authorisation letter and the rules of engagement.
What each of them is for is described in What makes a penetration test legal.
During the test
- Do not deploy to the environment under test without telling the testers. A finding that disappears overnight costs a day of confusion.
- Do not fix findings in the middle, except critical ones. If you fix one, say so.
- Answer questions quickly. A tester who asks how a function is meant to work has usually found something.
- Expect urgent findings at once. A critical vulnerability is reported when it is confirmed, not in the final report.
After the test
- Read the summary with management, the findings with engineering. They are written for different readers.
- Attend the debrief. It is the cheapest hour of the engagement.
- Plan the fixes by priority, and tell the testers when they are deployed.
- Use the retest. A fix that has not been verified is an assumption.
- Keep the report confidential. Until the fixes are in, it is a manual for attacking you. For customers and auditors there is the attestation letter.
The checklist
| Before the start | Done |
|---|---|
| Question and scope written down, exclusions included | |
| Environment chosen, test windows agreed | |
| Accounts created for every role, in two tenants if the product has tenants | |
| Documentation handed over | |
| Network access arranged and tested | |
| Test data in place, no real personal data | |
| Backups verified | |
| Operations, monitoring and providers informed | |
| Contact person named | |
| NDA, contract, authorisation letter, rules of engagement signed |
The page of each service lists what that particular test needs from you.