Why software security audits matter

This article was written by Jack Chan, a penetration tester at ExpressVPN.

ExpressVPN is first and foremost an online privacy company. To maintain our high standards of security, we frequently engage third-party auditors to verify our security claims. In the past two years, we’ve commissioned more than a dozen independent security audits covering all our apps, our Aircove router, our privacy policy, and more. These are in addition to our own rigorous internal auditing processes.

So what is the value of software security audits, and how are they conducted?

While all types of software undergo security audits, they are crucial for VPN services, which play a key role in protecting traffic and ensuring user privacy. Recent examples of major security incidents involving vulnerabilities in VPNs are salient reminders of the importance of strong security strategy.

Not only do VPN companies have to work hard to ensure security, but they must also be able to earn the trust of users. This is where third-party audits come in, validating our security claims in a transparent, independent manner.

What are security audits? Why are they important?

In general terms, most VPNs consist of two core components: the client used to create a connection (usually an application installed on a user’s device) and a VPN server or gateway. Both the VPN client and the server applications are software, written by the VPN vendor, and run on hardware (user device and server). 

Within the complex process of software development, mistakes and bugs are inevitable. Some bugs are small, like a button rendered in the wrong color. But some can be large, leading to serious vulnerabilities that can result in compromised devices and leaked data.

A security audit is the process of reviewing software and source code to ensure they are free of vulnerabilities. As part of quality control, a product should be subject to thorough security assessment before going to production for public use, to ensure that the software provided to customers is as secure as possible. 

How are security audits conducted?

Security audits should be done both internally and externally, and each follows a similar methodology. They share the goal of obtaining a full understanding of the software to remove any bugs. Although they have the same goal and similar methodology, they differ in some aspects:

Independence. External auditors are independent third parties, which act as fresh pairs of eyes. Additionally, reputable vendors report any security issues regardless of their own interests or relationships with the development teams. While our internal security team acts independently, verification by an independent third party is a key part of maintaining user trust.

Level of integration. While external auditors are completely independent, internal audits integrate strongly with the development life cycle, baking security in from the beginning, all the way from inception and design to delivery. Internal audits occur as part of the development lifecycle, while external audits usually focus on a finished product. We will detail how ExpressVPN approaches this in the section below.

Static vs. dynamic approach to security audits

Security auditors, both internal and external, usually take two approaches when auditing software: the static approach and the dynamic approach.

In the static approach, a security auditor is given a copy of the software, preferably with source code, and analyzes its structure, without executing the software. This is a critical part of a security audit, as it allows the tester to understand how the code works from end to end.

Next, in the dynamic approach, the auditor executes the software, understanding and analyzing its actual behavior in real time. Information from static analysis is also used here, and this is where the tester will validate that the security precautions taken are strong enough to withstand attack.

After the audit, a report with the details of each identified vulnerability is compiled, along with a set of recommendations. This report is shared with the product owners, allowing developers to review and implement the recommended fixes. Finally, the auditor will review the fixes as implemented to ensure correctness.

Not all security audits are equal in quality

Just like software itself, security audits have their own caveats. There are certain factors that can lower the quality of an audit.

Time limitations. While ExpressVPN always allocates sufficient time for security audits to ensure we maintain a high standard on our audits’ quality (i.e., we obtain coverage of the code base and features), security audits that we’ve observed in the industry are typically time-boxed, especially when resources are limited. The time needed for an audit depends on how complicated the software is, however having a shorter timeframe just because of limited resources reduces coverage: some areas of the software will be excluded from the auditing scope. These limitations prevent a thorough audit.

Auditor oversight. ExpressVPN always works with highly skilled and trusted vendors, and continues to evaluate the auditors we work with. It is critical to work with auditors who have deep knowledge and strong security testing skills; an auditor who is less familiar with the technology stack used by the software, or less technically competent in fully understanding the code, may miss potentially severe vulnerabilities.

We have a strong awareness of the limitations, and both our internal and external audit processes are designed to address these limitations. We will explain below.

Security audits at ExpressVPN

What makes our security auditing different? At ExpressVPN, we follow a set of well-developed workflows, designed to identify and address vulnerabilities at multiple stages during the software development lifecycle and ensure the quality of our audits.

Internal auditing

As the first step of our workflow, before any code is written, our security team plays an important role as design reviewers. We review the high-level design of the software being built, extensively model the threats it might face, and identify potential security risks, making practical suggestions to developers for improvement. This process helps us reduce structural weakness of our products and prevent potential risks before they become part of our code.

We continue to work hand in hand with our engineers throughout the development process to ensure we have a thorough understanding of the software before we audit it. When the application is nearly ready to be launched, we start an assessment and try to “hack” it. Our security team members, through their years of experience in the industry and exposure to a large variety of technologies, are experts in the practice of auditing, covering every platform our products run on, including Windows, Linux, and macOS clients, web applications, firmware, and Android and iOS clients. 

This same set of skills has allowed us to deliver stellar performances in “capture the flag” competitions—where “flags” are secretly hidden in programs or websites intentionally designed with vulnerabilities—at hackathons like Hack the Box CTFs. In fact, at last year’s Hack the Box, the ExpressVPN team placed eighth out of more than 650 teams.

An internal security audit is conducted by at least two members of our security team, who check the audit coverage daily and ensure important areas are thoroughly covered. As a result, we consistently identify almost all serious weaknesses long before any application goes into the hands of customers, and this is evidenced by the minimal findings during external audits of our products. To back this up, the full report of all external audits we’ve done over the past two years are publicly accessible to everyone—no paywall, no subscription, just transparency.

External auditing

Once internal auditing is complete and we’ve addressed all identified weaknesses, we work with our trusted vendors to independently verify the security of our products. We share extensive documentation with them to ensure they have the same understanding of the product as we do, and give them access to the source code so they can review what’s happening under the hood. 

As testament to the strength of our internal audit processes, relatively few weaknesses, and usually of low severity, have been raised by our independent partners—and these issues are available for all to see when our vendors publish the report. It’s important to note that these positive results are not typical within the industry, where it’s common to see a larger number of issues and ones that are of higher severity.

With all the effort mentioned, ExpressVPN is confidently serving our products to those who value security, and we will continue to maintain strong, transparent security processes.