What does HackerNews think of public-pentesting-reports?

A list of public penetration test reports published by several consulting firms and academic security groups.

Language: CSS

Hacking on a plane: Leaking data of millions and taking over any account | Dec 2022
Expand Context ↕
NCC Group is probably the biggest name because they go around Hoovering up companies that are usually above average in the competencies you asked about. And they can attract and retain talent.

Trail of Bits is another big name because they hire and retain talent across a large number of enterprise, emerging tech, and research verticals.

Other established firms include Atredis Partners, IOActive, Security Innovation. There are more one could list.

Sometimes these companies work with partners who ask to publicly disclose some artifact resulting from the test. Here is a collection of those reports aggregated by firm: https://github.com/juliocesarfort/public-pentesting-reports (Edit: note this is not a great way to evaluate any particular company, but it does provide an objective listing of companies that exist in the pentesting space).

Each firm will also have variability in their personnel for your project which can yield different results for two independent tests on the same target from the same firm.
Log4j: The pain just keeps going | Jul 2022
Expand Context ↕
I'd say don't let yourself be discouraged by GP. Just look into a company before you apply. Many have public reports and/or security research, both of which you could use as indicators.

Here's a repo with lots of public reports by various consultancies, you could use that as a starting point: https://github.com/juliocesarfort/public-pentesting-reports
Ask HN: How do security researchers know where to look for vulnerabilites? | May 2018
Since code audits are mostly conducted as an art rather than science with rigorous methodologies, I assume most vulnerabilities locations are just derived from experience and "poking around" work.

If you look at public penetration testing reports [3] seeing there mostly is no section about methodology, it's reasonable to assume that there are rather no true common standards or bodies of knowledge to find security vulnerabilities.

For some application security fields like web application security there are at least some semi-rigorous catalogs [1,2] which can help you to conduct more comprehensive code audits or security tests/audits.

As already mentioned there are already tools which can help you to conduct more professional and thorough code audits through static security source code analyzers or dynamic analysis tools (e. g. valgrind for memory related bugs or afl as an fuzzing tool example). These tools are focusing on implementation bugs, design weaknesses still have to be evaluated manually.

In my opinion the discipline of software security assessments hasn't grown up yet, but there is definitely research going on to improve the situation, e. g. [4] for a research example on finding bugs statically.

[1] OWASP Testing Guide v4: https://www.owasp.org/index.php/OWASP_Testing_Project

[2] OWASP Application Security Verification Standard (ASVS): https://www.owasp.org/index.php/Category:OWASP_Application_S...

[3] https://github.com/juliocesarfort/public-pentesting-reports

[4] Modeling and Discovering Vulnerabilities with Code Property Graphs: https://www.sec.cs.tu-bs.de/pubs/2014-ieeesp.pdf