Lecture
Modern web applications exist in an environment that differs markedly from the one that
existed at the dawn of the Internet. They have become important tools for the work of
companies, yet their operation often does not match what
browsers were designed for, whose engines were often created many years ago. Moreover, the growth
and expansion of web application functionality happened so quickly that it was not always
possible to achieve everything that was desired. This is especially obvious when it comes
to web application security. And although most companies use
certain protection mechanisms for their web applications, only some of them have
truly serious, comprehensive web application security systems.
This fact gives rise to two problems. First, the lack of a complete system significantly
increases risks and seriously worsens the consequences of potential attacks. Second, the lack of
a unified system significantly increases the cost of maintaining an acceptable level of
web application security, at which the risks and consequences of potential attacks will be
minimal.
This report presents information on how to build a pragmatic web application
security system at a reasonable cost while still providing effective
protection. Instead of delving into the specific details of any one technology, we
will identify the main components of a security system and how they should interact
with each other. We will begin by describing how web applications differ from
commercial and other applications. Then we will offer the main justifications that
can be used to obtain a budget for this task. We will also focus
on the specific security needs of web applications, delve
into the details of the main security components and, finally, combine them into a single web application
security system based on typical use cases.
The Web Application Security Problem
Commercial web applications have evolved in such a way that they have become a puzzle from a
security point of view. Although everyone in information security watched their emergence and development, in
the early stages they were classified as applications with a not very high level of risk. But in a short
time web applications grew from experimental systems into business-critical business applications. Ask any web application developer and they will tell you a story
about how their small internal project suddenly became a vital part of the business, as soon as
they made the mistake of praising it to their company's business unit. As a result, there is now
no way to halt the development of web applications and go back to basics to take into account all the
important points that were missed at the dawn of their formation, which ultimately led to the current
security situation:
• Before web applications came into active use, only a small number of companies
connected their internal systems with the outside world. And even those
that did so provided access only to business partners, and only a handful worked with a wide range of
users at all.
• Web applications grew on their own – they began as informational sites
that were only slightly more than catalogs. Then, through basic services, they were
connected to critically important back-end systems.
• Applications that were developed for a long time and were enriched with functionality for just as long
often inspire excessive trust, although in fact they may be
just as vulnerable as those that were made in a hurry. The classic
"boiling frog" principle applies here – if you drop a frog into hot water, it will immediately
jump out, but if you heat the water gradually, it will be boiled without even noticing it.
• Web application protocols were designed to provide simplicity and flexibility of
interaction, but at the same time they are largely vulnerable.
• The tools and methods for developing web applications are evolving very quickly, but still
rely on a large amount of legacy code.
• Threats to web applications evolve as fast as web applications themselves, and
in addition, threats regularly emerge against things that were built in the past.
The Web was originally not intended for securely running critical applications, which
means that all the applications developed actually use a platform unsuitable for this purpose.
• Only some applications were built from scratch with all security requirements in mind
and are also kept up to date for a sufficiently long time.
• When security incidents occur, there are problems with detecting and
analyzing them.
In light of all these factors, it is worth noting that funding for web application development
is, as a rule, insufficient. Moreover, like all developers, web application creators are under constant pressure, are strictly limited in time and are often
forced to turn a blind eye to some security-wise not quite correct
points for the sake of business goals. We categorize web application security problems
as follows:
• Only some applications are developed with security requirements in mind.
• The application depends on the web browser in use, which is neither a secure
nor a trusted environment.
• Web applications evolved to provide access to the most sensitive
back-end systems from a public network without a proper level of security.
• The Web was not designed as a secure platform and, accordingly, does not meet
the basic security principles – confidentiality, integrity, availability.
• Only a small number of security breaches are detected and allow
the company's losses to be analyzed and, accordingly, lead to increased attention to
the problem on the part of management.
• We have a huge amount of old code that is constantly being fixed, but at the same time
ever more new code is constantly being added.
• A web application is a complex of platforms, tools and services from many
providers, each with its own security challenges.
• Many web applications are critically important, but were developed without taking this
fact into account.
• If a web application is developed in-house, then vulnerabilities must be tracked and fixes released
in-house – no one else will do this work.
• Security requirements often conflict with the requirements put forward by
the business, in particular when it comes to ease of use and speed of operation.
• No single web application security tool will provide effective protection
on its own.
To sum up what has been written, we offer the following analogy: when securing web
applications, it is as if we were trying to protect a fighter jet in the middle of an aerial
battle with constantly changing functional requirements, parts, fuel and laws of physics. And all this while the jet is constantly attacked by millions of
diverse enemy objects about which we know practically nothing.
Application security has traditionally suffered from underfunding, and
convincing management of the need for new spending to improve it is quite difficult,
because it already works perfectly well. Of course, this applies not only to applications, but in fact
is a common problem for all of information security, where additional measures to improve the level of
protection do not generate enthusiasm in the business, since threats and the consequences of incidents are hard
to quantify. Building a complete model for justifying investment in
web application security goes far beyond the scope of this material, but we cannot
talk about creating a security system without at least some basic
tools that will let you determine how much needs to be invested and how to convince
management to support you. The following list is not an exhaustive business model
justification, but it includes the main drivers that can be used to
justify the need for investment in web application security:
Compliance. Like it or not, information security is
subject to government oversight and is also regulated by industry standards
and other requirements. We would like to divide the justification in this area into three
separate parts: the requirements themselves; additional requirements that may
arise (protection of personal data, etc.); reduced costs of compliance and
audit. On average, an organization uses all three factors to evaluate investment in security of
web applications.
Fraud prevention. If you can estimate the potential damage
from fraud with reasonable accuracy, this can be one of the important drivers when justifying investment
in security. In some cases you can obtain specific data on cases of
fraud and show how they could be avoided with certain
necessary solutions. Also keep in mind that you may not have the right infrastructure
for detecting and assessing cases of fraud, which in itself can be sufficient
justification. Penetration tests can also be a good chance of obtaining
investment to reduce the number of fraud cases – a test can reveal unknown
ways to exploit vulnerabilities, an active attack, or opportunities for future attacks. You can
use the results of a penetration test to assess potential opportunities for
fraud and draw up a plan that will reduce risks to an acceptable level.
Savings. As we already mentioned in the section on compliance, security controls
can reduce audit costs, but this is only an additional opportunity for savings.
Using web application security tools during development and operation
will reduce the costs of manual security checks and of fixing
identified vulnerabilities, and will also improve the overall efficiency of the web application.
We can also count on savings from a reduced number of incidents, as well as from
restoring web application functionality after them.
Availability. When working with web applications, we assess both the overall uptime of the
service and the availability of services (there is a chance of losing part of the application's functionality
because of an attack or the cleanup of its consequences). For example, it is extremely rare to see a complete
shutdown of a site due to web application security problems, whereas
the shutdown of some functions, including the payment system, is much more common.
Moreover, there are often cases when, during an attack, you have to disable some
functions yourself to protect users and their data, even if the attack does not
directly affect the operation of that service.
User protection. Although this parameter cannot be counted directly and cannot
be expressed as any sum of money, the main incentive for investing in web application security is the need to protect users and their confidential data,
since neglecting this can negatively affect their trust in the company and,
accordingly, its image. Attackers quite often do not cause any
damage to a compromised site, but use it to attack users. That is, even
if you lose nothing as a result of an attack on your organization, it remains a problem,
because your web application can become a platform for attacks on your users. Most
companies receive direct or indirect revenue from user data, and this already
creates an obligation to protect that data.
Reputation protection. While many models try to quantify
reputation and estimate the potential losses from its deterioration, the reality is that all these
models are fiction – there is no exact method for measuring the potential damage to reputation from
attacks by malicious actors. Despite numerous forecasts about the
likely consequences of information leaks and the impact of such an event on reputation, most
of them do not come true. And sometimes the opposite happens. For example, after the large retailer TJX
suffered one of the largest data breaches in history and this fact was made public, sales
went up. But the mere fact that reputational risks cannot be calculated precisely does not
mean that they are not an important factor when justifying the need for investment
in web application security. Simply ask yourself (or the company's management) how
important the application is to the company's image and how prepared you are for the risks associated
with the loss of customer information. The trust of users, investors and partners in your
company and services, although it does not depend directly on the security of the web application, is nevertheless
very important and can affect the overall value of the business.
Direct losses from incidents. Besides losses from fraud, there are also
direct financial losses. Even if we ignore the reputational
losses of the business, as well as the value of the company, and focus only on losses that
can be expressed in concrete figures, it turns out that these exist too. For example, the cost of sending
notifications to users, restoring operations and paying call center staff.
You will work out which combination of these works best based on the needs of your company
and management's priorities, but you should combine qualitative and quantitative
assessments in order to set investment priorities correctly. If you work with
personal data (finance/retail/healthcare), then security controls and cost reduction
will be the best arguments. For general-purpose web applications, important factors will be
protecting the user and reputation, reducing fraud risks and ensuring availability.
And let us not forget that all these arguments also apply to internal
applications. Regardless of the field of application, there is no shortage of business justifications (unlike the technical side)
in favor of investing in web application security.
We have spent many years studying network and host security issues and have built
information technology security systems focused on the underlying
infrastructure. But, as we have already noted, web application security differs markedly
from all of this and requires a completely different approach. Web application security differs
from the protection of traditional software as well, although it has much more in common with it. In the previous
section we focused on the business, and now we will move on to the technical side and talk about
specific technical and non-technical differences in web application security, before
we give our recommendations on the web application security lifecycle.
Why securing web applications differs from securing networks
and hosts
When securing a network or a host, we focus on blocking
user capabilities within the boundaries of software, devices and systems.
Likewise, when securing enterprise applications, we mostly deal with
locking down platforms, protecting communications, authenticating users and
enforcing security controls through the means provided by the application
platform. But with web applications we face not only all of these
issues but also have to deal with third-party or in-house code. Depending on whether
the application is accessible from outside or intended only for internal use, they
differ significantly:
Browser UI: In most applications we build the user interface ourselves. But with
web applications we rely on a third-party viewer (the browser), which we cannot
fully control and which may interact with other applications at the same
time.
Custom code creates custom vulnerabilities: With web applications you usually
write the code yourself (using plugins and frameworks along the way). This means that most
of the vulnerabilities in your application will be unique. If you do not monitor
your application, no one will tell you about any existing vulnerabilities, and no one will
offer a patch to close them.
You are the developer: When a vulnerability appears, there will be no developer to prepare a patch
for you (you must, of course, install patches for all the infrastructure components, frameworks
and scripts that you use). If you serve customers, you need to
comply with the service level agreement clause and ensure the required availability of the
service.
Firewalls alone cannot protect a web application: When we discover
vulnerabilities in our enterprise software, operating systems, applications, databases and so on,
we use tools such as firewalls or IPS to block the opportunities
for a potential attack while waiting for a patch to be released for the vulnerable software.
Such "blocking until patched" is only of limited effectiveness. And a Web Application Firewall (WAF) cannot protect you from logic errors. Although a WAF can help with
some classes of attacks, out of the box it does not know or understand your application,
and therefore cannot "cover" custom vulnerabilities. A WAF is certainly an important
part of web application protection, but only if it is part of a comprehensive
system, which we will discuss later.
Eternal beta: When a traditional application is developed, it goes through a full cycle of
checks: controls at the development stage, internal testing and, often,
additional testing involving outside users, that is, beta testers. And
all of this happens before the application goes into production. It would be great if web applications also went through this sequential cycle, but, as already mentioned, this happens
extremely rarely. Most web applications, even those their developers regard as
unfinished, are in fact already in production and have even become key
solutions critical to the business. Other applications are in a state of
constant change, and their developers do not even follow a formal development
cycle. Constantly changing applications are a difficult challenge for existing
security controls such as WAFs.
Reliance on frameworks/platforms: Web applications are rarely built from scratch, delighting us with polished
C code. Most often they are a mix of assorted frameworks, development tools and platforms
that were not always designed to work together. It is up to the developers to ensure
that all these parts, as well as their own code, work together. This approach often creates
serious security problems because of improper use of the components, their
interaction with one another, and with the custom code.
Legacy code: Even if new code is written from a clean slate and is truly secure, then
do not forget that in most cases there is also a huge body of old code,
which likewise has a huge number of vulnerabilities and therefore must be analyzed and fixed. If
old code is in active use, it must be just as secure as
new code. Often it is precisely outdated code that becomes the culprit of attacks on a web application.
Dynamic content: Most web applications are extremely dynamic
in nature, creating most of their content "on the fly", often using data (including
code) supplied by the user. And the browser tries to process all of this, thereby creating
new classes of security problems.
New classes of vulnerabilities: As with classic applications, researchers
and attackers are constantly discovering new classes of web application vulnerabilities. There are many such
examples. Therefore, remember that even code that is perfect by today's standards
does not guarantee that it will always remain secure.
We have listed a number of reasons why web applications should be viewed differently than
classic ones. Implementing web application security requires reaching
consensus in the company. Making changes will be difficult and risky. In the long run it
will lead to lower costs, but all the changes must be paid for up front, without knowing to what extent
this investment will pay off. That is, to convince management of the need for
investment, you will need weighty arguments. And if you compare the costs with the benefits
described in this section, you will get a fully objective answer as to how these security expenses
will affect your company.
In the rest of this material you will find recommendations on effective steps
that help raise the level of web application security. Given the scale of the task,
we cannot give detailed information on every component of a security system,
and will instead consider them in general terms. While web applications pose challenges different from the usual
ones, the additional steps for protecting them do not differ from what you do now. We have broken the simplified
scheme of the development and usage cycle down into 3 steps and 7 zones:
Secure development
Process and training: In this section we focus on building security into the software
development lifecycle. This includes, among other things, training the people
involved in development and improving the development process. Developer training should be
oriented toward ensuring the security of the application's functionality.
We will also look at tools that help automate part of the work:
static analysis to find vulnerabilities in code and dynamic analysis to detect
anomalous application behavior.
• Secure development: Introducing secure development practices into the process of creating
a web application.
• Static analysis: A tool for scanning an application's source code for security errors
. These tools are also called "white box".
• Dynamic analysis: Tools that examine not the source code but the running
application itself while attempting attacks. These tools are often called "black box".
At the stage when application development is already finished or, at a minimum, it has been brought to a point
where it can be used for testing, it is time to verify that the
application is free of known security problems and is configured properly. At the same
time, you can begin vulnerability assessment and penetration testing,
along with the traditional configuration analysis and checks of consistency of interaction with
other systems.
• Vulnerability assessment: Remote scanning of the web application, both with an account
and without one. A web application vulnerability assessment concentrates on the application itself,
whereas a standard vulnerability assessment focuses on examining the host platform.
• Penetration testing: Penetration testing is effectively an
attempted break-in that reveals security weaknesses and the
risks they carry. Penetration testing makes it possible to assess an application's flaws,
classify them and prioritize them correctly.
Most people do not pay due attention to testing their own code, and
focus exclusively on expanding functionality, adding new features
and achieving stable operation of the web application. All of this is, of course, important, and a serious
team of developers is assembled for it. The approach to security should be just as serious,
with the code thoroughly checked and every new feature added to the
application subjected to a full security analysis.
Secure usage
In this section we move from the tools and processes that make an application
secure at the development stage to those that provide capabilities for detecting attacks
and responding to them. The main emphasis here is on the capabilities of a WAF, which should
protect the web application from unauthorized user actions, as well as on other
tools that scan requests and detect inappropriate activity directed at the
web application and other components related to it. Recent developments in the area of detection
tools help enforce policies, respond adequately to events, and
also combine several services into a cooperative hybrid model.
• Web Application Firewall: A network tool that monitors traffic to the
web application and reports known attacks and/or blocks them.
• Application and database activity monitoring: A tool that monitors
application and database activity (using various methods) for auditing and
generating security alerts based on violations of defined rules.
Since many organizations are only beginning to create application security policies,
we will try to outline a framework and illustrate the capabilities, which will make it possible to consider
the overall picture of an organization's web development. It should be stressed that we are only providing
an overview of the available capabilities, along with basic information on how it works and how to
integrate it. More detail, especially about the "how", cannot be given within this material
– this topic deserves several full-fledged books. The specifics of modifying processes,
applying various methodologies and the related nuances are also beyond the scope of this
document.
Web application security is undergoing very rapid change – changing as quickly
as attackers find new ways to attack. We will discuss advanced tools and
the future of the market, as well as the specific tools used to identify and protect
against new kinds of attacks. While tools are very important, they cannot be used
alone. You need the right tools in the right hands as part of the right
process. Nevertheless, we will devote a significant amount of time to tools, for the
following reasons:
• Many tools automate most of the advanced approaches to security
assessment
• Your developers and quality control staff may be unable to keep up
with constantly changing information security trends, and may have
insufficient expertise in this area, and tools will help to compensate for these shortcomings to a considerable
extent.
• The scope of security testing work needed to ensure proper quality
requires automation. The ratio of requirements to resources for any working group without
automation would be very, very poor.
• Tools provide a platform for configuring and testing policies and are
the result of research and enormous experience, which makes them indispensable at all stages of
development and use of a web application.
Going forward, we will examine each of the described parts and show how they fit into the
overall web application security system, and point out their strengths and weaknesses.
Naturally, we cannot give a complete description, since each of the sections deserves a separate
book, and so we will concentrate on the main points.
Web application security often receives insufficient attention in the processes of
modification, training and the choice of development tools. Security very often starts to be addressed
after the main development is finished rather than during it. Yet attention
to these issues during development makes it possible to make an application more secure at lower
cost. In this section we will discuss the best options for integrating security into a web application at the development stage.
Web application security: preparation and the application development
lifecycle
Most of today's web applications were designed, developed and deployed
before norms and principles of web application security appeared. Secure development methods
in web development teams often enter their awareness only after some incident
occurs. Management, for its part, often thinks about security only
within the framework of existing requirements whose fulfillment is mandatory. The news often draws
attention to attacks using SQL injection, but most developers have no idea
what cross-site scripting attacks do, and even fewer know
how to defend against them. The practice of secure application development is in its infancy
and is far from maturity. Of course, significant knowledge has already been accumulated, but it is not yet
comprehensive and has by no means reached every developer.
Whatever your motivation, education and process modification are the most
important first steps toward creating a secure web application. If you are developing a
new application or modernizing an old one, project managers, developers and other
specialists must be aware of the potential security problems of the solution, as
well as of methods of secure design and coding. The training program should
cover both general threats and the known methods that attackers use to
attack systems. Specialized training is needed for staff in each subdiscipline,
including process modification capabilities, the secure platform deployment model,
security tools and testing methodology. If your team does not know how to
provide a proper level of security, you will depend on third parties for it
(service providers or hackers), and that costs far more.
Project managers must know which types of threats may apply to the
web application being developed and how to minimize existing risks while
providing the required functionality. Developers must understand how exploits work,
how to test for them and, of course, how to fix the flaws. Understanding is the key to
building secure applications, so it is very good if your company invests in
employee education by organizing in-house training, holding educational
events, or paying for training at outside training centers. Threat modeling,
secure development principles, attack surface reduction, functional
security requirements and testing are the core of secure application development principles and
integrate easily into all development processes and stages.
Process also plays an important role in application development and affects security just as
it affects employee productivity and product quality. If the product specification
does not contain security requirements, you cannot hope that the product will be
secure. A product that is not security tested, just like a
product that does not undergo functional testing, will suffer from flaws and
bugs. A modification of the Software Development Lifecycle that includes security requirements is called Secure SDLC and includes simple
checks throughout the entire development process in order to identify problems as
early as possible. Secure SDLC is a serious enough topic for in-depth discussion, but the main
thing we would like to convey is the need to build security requirements into
every stage of development.
The tools and test cases that we will discuss later can be used to
automate testing and control, but knowledge and education are essential for using them
correctly. They can improve the development and testing process,
and reduce costs compared with bolted-on security, while reducing the
number of vulnerabilities in code. Team members with security education
are able to build libraries of tests that help detect typical vulnerabilities in new
versions of the code. Moreover, these are suitable not only for internal testing at the development stage
but also useful when testing an application that is already running. Extreme programming
techniques can help confirm that modules and components meet security
requirements during testing, along with checking functionality
not directly related to security. Remember: you are the developer, and your team should
know the code better than anyone else, including its flaws.
There are a number of purpose-built third-party tools that
help check code for security. Static analysis examines the source code of a web application to identify common vulnerabilities, errors and omissions within the
constructs of the programming language itself. It is, in a way, an automated analogue of
expert review. Among other things, these tools are mostly used to
scan for unhandled error conditions, the existence of an object and/or size determination, and
potential buffer overflows.
These tools are used at the development stage to identify flaws before
moving on to more formalized testing procedures. After all, the sooner a problem
becomes known, the easier it is to fix. Static code analysis is performed by the developers themselves,
which significantly speeds up finding errors and makes it noticeably cheaper than
if separate people were doing the analysis. These tools can be integrated with
source control to run the analysis automatically, again, in order
to identify flaws in the code as early as possible.
Static analysis is effective for detecting problems directly related to
coding errors that programmers make. The best tools integrate well
with various development environments (providing additional information about the detected
code flaws and offering options for fixing them), prioritize
the discovered vulnerabilities based on defined criteria, offer detailed reporting
and trend tracking, and are able to interact with the security team
involved in the development process without distracting other developers.
But static analysis tools do not account for all paths in the code, and cannot detect
certain types of vulnerabilities that appear only while the application is in use.
To fill this gap, over the last few years tools
for dynamic and hybrid analysis have been actively developing.
Dynamic analysis tools
Dynamic analysis is used to identify problems and vulnerabilities that cannot
be found by analyzing the source code or that are much easier to spot while
the application is running. One type of analysis is called "fuzzers", which send
the application deliberately malicious or bogus data
and monitor the results of these actions, as well as crashes of the application. Many
different variations are available on the market, but in general they can be divided into three groups:
White Box or Black Box. The testing tool may have no prior knowledge
of the application and study it as a "black box" (Black Box) with the corresponding
approach. But in most cases the "White Box" approach is used, where testing
looks for suitable vulnerabilities according to the features of the application
known to an authorized user. "Black Box" testing is more
representative, since it replicates how an external hacker works, who is unfamiliar
with the internal organization of the application. But when testing web applications both
approaches are important. After all, many vulnerabilities may be invisible to an unauthorized user but
perfectly visible to someone who has logged in.
Sending requests. Requests can be random, generated on the fly based on responses,
or targeted. Random requests are a good opportunity to check the
effectiveness of integrity checking and to find common flaws in the code, while targeted requests can be
used to check for already known vulnerabilities.
Output data and post-testing actions. After testing with the
White Box and Black Box methods, expert review of the test results is required by specialists who will separate the real flaws from
false positives. Error conditions are fairly easy to find,
and most dynamic analysis tools detect and report their
presence. Some monitoring tools not only detect errors but also indicate
which part of the code they are in. Like debuggers, a dynamic analyzer can
monitor the internal resources of an application while it is being tested,
by observing memory, pointers, request queues and variables. The tests can reveal
specific effects of the system usage model and problem areas.
Partly because of these variables, dynamic analysis tools differ in speed,
effectiveness and automation. Because they focus on the behavior of the application, they
give concrete results for "what if..." scenarios, which static tools
cannot. Dynamic and static analysis are complementary technologies, but they are
intended for different audiences. Some vendors offer both capabilities in a single
tool, which allows a more comprehensive assessment.
A well-structured web application security program begins with good
education and the integration of security into the application development life cycle – with
threat modeling, secure design, consideration of security in functional requirements,
proper use of static and dynamic analysis, use of secure
coding practices, continuous training and formalized testing.
Keeping the narrative consistent, we shift the focus of our attention from development
to the deployment of the web application. At the deployment stage, after the quality and
security of the application have been verified, we move on to vulnerability assessment and penetration testing
to make sure that our application meets current requirements and is
resistant to attacks. Remember that we look at web application security as a
continuous process with overlapping stages. Although we divide the process into separate
stages to make the narrative clearer and more structured, this does not at all
mean that one stage ends when another begins. For example, you should continue
to use dynamic analysis at the deployment stage, and also continue searching for
vulnerabilities and running penetration tests at all stages. Also remember that we are showing you the
big picture and the available options. We will discuss prioritization and the areas worth focusing on
particularly if your budget is limited a little later – after all, we are not
naive enough to think that you can afford all the solutions available on the market.
In a vulnerability assessment, we scan the web application to identify any points
that could potentially be used against us by attackers (we also assess
compliance with requirements and standards, but we focus specifically on finding vulnerabilities).
Scanning can be performed using tools, services, or a combination of the two.
Vulnerability assessment of web applications is very different from a standard vulnerability assessment,
which focuses on networks and hosts. In that case we focus on port scanning,
connections to services, and use other methods to gather information and determine
patch levels. As we have already said, even "standard" web applications are in many respects
custom-built, and therefore we need to examine them more carefully, checking the functions and
logic of the application, and also use custom checks to identify the vulnerabilities of the
web application. With a large amount of custom code, we must rely less on
searching for standard vulnerabilities and try to conduct more tests by way of attack. As we have already
said, custom code creates custom vulnerabilities.
When assessing the vulnerability of web applications, we skip the traditional host and network tests and
focus on third-party web applications and frameworks, as well as on the technical and business
flaws of custom applications.
The market for web application vulnerability assessment solutions offers both dedicated
tools and services. If you choose the path of self-testing, then, besides
selecting the right tools, you must be sure that they will be in the hands of an experienced
operator who can correctly interpret the test results. It is important to use
both testing methods, where the operator has different levels of access to the web application and
where the operator has none at all.
There are a number of commercial, free and open-source tools
and solutions for web application vulnerability assessment with broad functionality. Some
tools use a very limited set of exploits, and therefore experienced testers
use not one tool but a whole set of them, supplementing it with manual checking methods.
For example, there are applications focused on finding and testing only vulnerabilities
related to SQL injection. Enterprise-class tools should be more
multifunctional and cover a number of vulnerability classes critical for a web application,
such as SQL injection, cross-site scripting, etc. The OWASP Top 10
is an excellent baseline list of the main vulnerabilities, but enterprise-class applications
should not limit themselves to checking against only one list or category of vulnerabilities.
An enterprise-class solution must also be able to test multiple
applications, support tracking over a long period of time, and provide
adequate reporting (especially with regard to compliance), and also
be configurable for assessing compliance with local requirements.
Vulnerability assessment tools are, as a rule, software, but may also have
a hardware component. They can also work either under operator control or perform
scanning automatically on a schedule. Because web applications change
quite actively, it is very important to scan them after any changes are made or new
versions before deployment, and also to monitor live applications on an ongoing
basis.
Not all organizations have the resources (including expertise) to assess the vulnerability of their own
applications. In that case there are two options – buying the necessary tools or an external
assessment.
There are three main categories of web application vulnerability assessment services:
Fully automated scanning. Fully automated scanning without
operator involvement. The cost of such an assessment is minimal, but it produces many more errors. Such
checks are quite effective for continuous monitoring, but their capabilities are rather
limited for pre-deployment checks.
Automated scanning with results processed by an operator. In this option,
most of the checks are performed automatically, after which the results are
interpreted by a person who conducts additional tests as necessary.
Compared with a fully automated check, this option gives a deeper
investigation and more accurate results, but naturally costs more.
Manual examination. In this option the assessment is carried out by trained specialists
who use their own tools to find vulnerabilities and verify them,
after which they provide detailed reports. This option is the most effective and makes it possible to
identify more vulnerabilities than the previous ones, but it is also the most expensive.
Penetration Testing
The goal of a vulnerability assessment is to find the paths that potential attackers could
use, while a penetration test is an even deeper investigation that makes it possible to
understand whether the vulnerabilities found really pose risks to the company in the event of an attack. In
a penetration test we try to attack our applications in the same way as an attacker would,
in order to understand what the consequences of these actions might be.
Vulnerability assessment and penetration testing complement each other perfectly and are often
used together, since the first step in any attack is finding vulnerabilities that could
be exploited. The goal at the vulnerability assessment stage is to find the flaws of the
web application, while in penetration testing these flaws are checked in terms of
whether they can be exploited, the potential damage from each of them is assessed, and priorities are set
for fixing them.
The key value of a penetration test is that it determines the link between a vulnerability
and the risks it carries, and allows an informed decision about the need for and urgency of
fixing it. A vulnerability assessment provides only information about the presence of security gaps, but
does not give the big picture of the potential consequences of exploiting them. For example,
during a vulnerability scan you found that a web application is vulnerable to SQL injection, but when
you attempt to exploit this vulnerability it may turn out that an attacker can use it to
obtain confidential information, alter functionality, or even take the
web application down. It may turn out that minor vulnerabilities can lead to
an attacker gaining full control over the entire web application.
Some experts believe that penetration tests are important because they use the same
methods and pursue the same goals as a real attacker, but we believe that these tests are
very useful as a tool for prioritizing risks. However, both of these goals of the
tests are important, and one cannot be abandoned by giving priority to the other.
As with vulnerability assessment, penetration tests may not be limited to the
web application alone, but may also include checking for vulnerabilities in the environment – operating systems,
networks, user applications, etc. Tools and services in this area are available on
the market. But, given that we are talking about web applications, we will not delve into the environment
and infrastructure, and will limit ourselves to discussing the aspects of penetration testing
specific to web applications.
Penetration testing solutions make it possible to identify and verify the exploitability of
web application vulnerabilities, and to assess the possible consequences of attacks
that use them. Penetration testing specialists use many
different tools in their work, but most of them are highly specialized and do not
offer broad functionality. They are run before the web application is deployed, using
test data, since exploiting some vulnerabilities can lead
to damage to the stored data or to the application itself. Enterprise-class solutions
have additional features that help manage risks more effectively and
have an internal vulnerability scoring system, which reduces costs when conducting
internal penetration tests. They also usually cover a wider range of vulnerability
classes, support testing methods that are "safe" for the data and the web application,
which allows them to be used on live applications, and provide extensive
reporting and automation of ongoing assessments. One of the advantages of penetration testing
solutions is that they can be used not only to check an already
running application, but also at different stages of web application development and deployment.
Tools and solutions always come in the form of software and must
be used only by trained specialists. After all, even though a significant
part of the process is automated, only a well-trained and experienced specialist
can analyze the results of automated testing and effectively investigate a
vulnerability, drawing conclusions about the consequences of its possible exploitation.
When using penetration testing tools (just as with vulnerability
scanning tools) you have the option of running them in a safe
mode, which noticeably reduces the risk of problems with a running web application.
But if the application is really running in "production" mode, be careful even when
working in safe mode, since even in this case there is a certain risk of disruptions to
the operation of the web application, as well as of cluttering the databases. Therefore such testing must be
carried out under maximum control.
Unlike what we saw when studying vulnerability scanning, there are no fully automated
penetration testing services. Because of the flexible and constantly changing nature of
web applications, penetration tests always require human oversight. Penetration
testing services are offered either as one-off services or as a
subscription, which implies conducting tests at a certain frequency. The frequency of
testing varies depending on the needs of the business, and the criticality and purpose of web applications. Internal applications may be tested only once a year as part of regular
infrastructure testing.
Before using penetration testing services, you must carefully
examine the experience, knowledge and capabilities of the service provider. Some companies offer
little more than remote scanning, but are unable to provide adequate
results that you can effectively use in risk management. Others simply
stop after they gain access to the application using a single vulnerability,
without conducting a full check. Therefore, when choosing a service provider, you should give preference to
organizations with a proven process that can provide sample reports and will
meet your expectations. Most penetration tests are structured
and have a fixed time and depth of investigation. In the case of fixed time
(the most common option), testers try to find as many vulnerabilities as
they can within a period limited by mutual agreement. The work may also be
divided into phases, for example: blind search, search under the account of an authorized user,
etc. Fixed-depth testing implies stopping the testing only
when a certain goal is reached (for example, obtaining administrator rights or access to
payment card numbers) and does not depend on the time spent.
Integrating Vulnerability Assessment and Penetration Testing into
Secure Deployment
Although most companies focus on vulnerability assessment and penetration testing
of already running external web applications, in reality this process
should begin at the deployment stage and should not be limited to only those web applications that are accessible from outside. It is very important to test the application before
you give external or even internal users access to it. Also there is
no reason, other than perhaps resources, not to incorporate risk assessment and penetration testing into
the application development process at the main stages.
Before the web application is deployed, a test environment is created that is identical to the one
in which the "production" system will run. All of its elements, including product versions and
connections to external services, must be identical. Then a vulnerability assessment is carried out
and penetration testing is performed. If you use your own tools, you must
also have trained staff who know how to use those tools. If you use
the services of a third-party company, you need to provide remote access to the test environment
– study and meet the access requirements that contractors have. Not every company
can provide full and comprehensive testing of each of its applications, so
you need to prioritize your list of web applications based on the criticality of the
application to business operations, the application's use of confidential data,
etc.
For some large and especially important web applications, you could conduct a vulnerability assessment
and penetration testing with third-party companies even before deployment begins. If
your web application is business-critical or intended for public access, such a check
is simply essential.
After the web application has been deployed, your task will be to perform at least
basic checks whenever any changes are made to its code, before the update
is installed on the "production" system. For business-critical applications, periodically
use external vulnerability assessment and penetration testing services (at least
once a year, or when major updates are released) in addition to conducting your
own tests.
We know that different companies have different resources for maintaining up-to-date
testing tools and third-party services, and therefore you need to adapt
these recommendations to the realities of your organization. We have provided only fairly general
information aimed at achieving the best result, along with some data that will help you
set priorities correctly.
In the previous parts we talked about secure development and secure deployment,
and now it is time to move on to secure use. This stage begins when
development and testing are fully completed and the web application enters active
use. Remember that most of what we discussed in the previous
stages will be useful to you at this stage as well, so do not rush to throw away the tools you used
and forget the skills you have acquired. Updates to the web application should still
be developed securely and undergo comprehensive testing before installation,
vulnerability assessment and penetration tests should also be regular, and configuration management and
continuous security monitoring become even more important than they were
before.
In the secure use phase we add two new categories of technologies to support
two new processes – the Web Application Firewall (WAF), as a shield repelling various
types of attacks, and web application and/or database monitoring solutions, which will provide
oversight and notification of security threats.
Web Application Firewall (WAF)
The task of a WAF is to sit in front of the web application or alongside it, monitoring
application activity and warning of security threats or blocking
them. Thus a WAF performs two functions – activity monitoring in the web application and
preventive security control.
A WAF is a firewall that is originally designed with a view to
controlling HTTP requests and blocking those that may be potentially dangerous
or do not conform to the specified rules. The task of a WAF is to intercept SQL injection,
cross-site scripting (XSS), directory traversal attempts and other attempted attacks via
HTTP requests, including request spoofing, that is, to stop any attempts at unauthorized
use of the web application. The rules and policies set in a WAF make it possible to effectively
control HTTP traffic in accordance with the functionality of the web application. A WAF
warns of or blocks suspicious activity on the basis of existing signatures (of already
known attacks) or on the basis of the specific requirements of the web application.
Modern WAFs examine incoming and outgoing requests, compare them with the specified rules
and issue alerts if any deviation is detected. Based on the analysis of
suspicious requests, a WAF chooses one of four courses of action: 1) let the
request through, 2) let it through but record the fact of non-standard behavior, 3) block the transmission of the
request, 4) reset the connection.
A WAF is usually a network device. As a rule, it is placed directly in front of the
application (proxy mode) or off to the side of the application, mirroring to it the traffic exchanged by
the web application. In the first installation option, all incoming and outgoing requests are
intercepted and checked before they reach the web application or the user, which
reduces the load on the web application. If an SSL connection is used to communicate
with the web application, the WAF installed in front of it must be able to decrypt it
and analyze it. When the WAF is installed off to the side of the application, the problem of working with SSL
traffic is solved by giving it a copy of the server certificate, or by installing it after an
SSL concentrator. Some vendors also offer WAFs as plugins for specific
platforms, that is, ones that have no hardware base of their own.
The effectiveness of some WAFs is limited by the quality of their policies. Policies are important not
only for detecting and blocking known types of attacks, but also for a flexible response when
unknown types of attacks are detected, while preserving functionality for normal requests.
The complexity of web applications, combined with the need to update policies constantly
and the various deployment options, is a difficult task for all WAF developers.
At the same time, simply installing a WAF in front of a web application and enabling all of its features is
a sure recipe for disaster. There is no chance that a WAF will understand all the peculiarities of your web application on its own, and therefore its deployment requires careful and attentive configuration in order
продолжение следует...
Часть 1 Web Applications: Secure Development, Deployment and Use
Часть 2 Summing Up - Web Applications: Secure Development, Deployment and Use
Comments