The goal of the exercise is simply to find all of the security vulnerabilities that exist in the systems being tested. In this context, a vulnerability is anything that increases the likelihood that an attacker can disrupt or gain unauthorised access to a system and any data contained within.
The most common vulnerabilities tend to be design flaws, configuration errors, and software bugs. These get introduced during development and implementation, generally by accident, and once identified by the penetration testing, can usually be quickly resolved by a little re-engineering.
Most organizations will have a penetration test carried out due to one of the following reasons:
Some industries and types of data are regulated and must be handled securely (like personal data, the financial sector, or credit-card data). In this case your regulator will insist on a penetration test as part of a certification process.
You may be a product vendor (like a web developer) and your client may be regulated, so will ask you to have a penetration test performed on their behalf.
You may suspect (or know) that you have already been hacked and now want to find out more about the threats to your systems, so that you can reduce the risk of another successful attack.
You may simply think it is a good idea to be proactive and find out about the threats to your organization in advance.