I don't think the limited time window is the issue. You cannot give a stamp of approval for the future. You can state that you found the network/application was PCI compliant as per the guidelines during that test window, but that's about it.

There are many other issues that make PCI compliance complete crap.

1. The vetting process for ASVs is completely ridiculous.
[The applicant assessing the PCI test environment is scored based upon the known vulnerabilities identified within the test environment. This test environment is a set of servers installed on vmware. Should one of the vmware hosts fail during the enumeration portion of the test, this is *not* taken into account. This host *cannot* be listed as a failing host in the report. The applicant will lose *all* points associated with that host. If there were 30 vulnerabilities that the applicant was supposed to find on that host, and 1 was found because the host fell over, then the applicant will lost credit for all the "missed" findings. Never mind that in a real world scenario this host would have failed the test and been marked as failing in the report. Furthermore, it would have required attention to fix and make it complaint. And isn't that the real goal, fix the security issues?]

2. The PCI ASV guidelines are about as straight as the Earth's magnetic field.
[I don't have the newest ASV documents, but version 1.1 of the Security Scanning procedures is mostly a joke. It all starts with the scope. If he client gives you range 10.x.x.x and you find host 172.16.x.x, the SSP 1.1 dictates that you validate *with the client* that this host should be within scope. Forget that you know it is the SQL backend to the web server. If the client says it shouldn't be scanned, it's not in the report. This is a great loophole that was used often by a few clients I know to avoid having to fix applications that would cause them to fail. The rest of the requirements for an ASV were laid out in the PCI DSS Technical and Operational Requirements for Approved Scanning Vendors v1.1. This document states the same bunk about asking the customer if those IPs found are within scope.

One of the specifications was for an exhaustive fingerprinting scan on all TCP and UDP ports. When asked for clarification, PCI drones said something along the lines of "a SYN scan should be sufficient". SYN + UDP == ??? Forget about asking how a SYN scan == fingerprinting.

And my all time favorite is the: "Custom Web Application Check" section. Which states: "The ASV scanning solution must be able to detect the following application vulnerabilities and configuration issues: Unvalidated parameters which lead to SQL injection attacks, Cross-site scripting (XSS) flaws." Now, I don't want to point fingers (OK, I do, but I won't) but how many "scanning" solutions can you name that can crawl and find all SQL and XSS vulnerabilities in a web app? Yet, these "scanning" solutions have been PCI ASVs for a long time (some had been working on more app-centric type tests during the process of laying out the new ASV requiremets). Some of them wouldn't even crawl an app correctly let alone find any flaws. Keep in mind, these are the requirements from 2006. Anyone remember "scanners" that did network level vuln assessment and also did a proper web app spider and search for SQLi and XSS in 2006?]

3. Failure to comply isn't often punished. There is no positive nor negative reinforcement applied to companies that pass or fail an ASV assessment. The fines are almost always pushed off the first time. Giving the company a chance to remediate any issues. However, the chances at remediation seem to go on for a few months, if not a year or two. All the while customer credit card data is at risk.]

I could go on, but the comment is big enough as is. :)