Nmap's algorithm for detecting matches is relatively simple. It takes a subject fingerprint and tests it against every single reference fingerprint in nmap-os-db.
When testing against a reference fingerprint, Nmap looks at each probe category line from the subject fingerprint (such as SEQ or T1) in turn. Any probe lines which do not exist in the reference fingerprint are skipped. When the reference fingerprint does have a matching line, they are compared.
For a probe line comparison, Nmap examines every individual test (R, DF, W, etc.) from the subject category line in turn. Any tests which do not exist in the reference line are skipped. Whenever a matching test is found, Nmap increments the PossiblePoints accumulator by the number of points
assigned to this test. Then the test values are compared. If the reference test has an empty value, the subject test only matches if its value is empty too. If the reference test is just a plain string or number (no operators), the subject test must match it exactly. If the reference string contains operators (|, -, >, or <), the subject must match as described in the section called “Test expressions”. If a test matches, the NumMatchPoints accumulator is incremented by the test's point value.
Once all of the probe lines are tested for a fingerprint, Nmap divides NumMatchPoints by PossiblePoints. The result is a confidence factor describing the probability that the subject fingerprint matches that particular reference fingerprint. It is treated as a percentage, so 1.00 is a perfect match while 0.95 is very close.
Test point values are assigned by a special MatchPoints entry (which may only appear once) in nmap-os-db. This entry looks much like a normal fingerprint, but instead of providing results for each test, it provides point values (non-negative integers) for each test. Tests listed in the MatchPoints structure only apply when found in the same test they are listed in. So a value given for the W (Window size) test in T1 doesn't affect the W test in T3. A test can be effectively disabled by assigning it a point value of 0. An example MatchPoints structure is given in Example 8.7.
Example 8.7. The MatchPoints structure
MatchPoints SEQ(SP=25%GCD=75%ISR=25%TI=100%CI=50%II=100%SS=80%TS=100) OPS(O1=20%O2=20%O3=20%O4=20%O5=20%O6=20) WIN(W1=15%W2=15%W3=15%W4=15%W5=15%W6=15) ECN(R=100%DF=20%T=15%TG=15%W=15%O=15%CC=100%Q=20) T1(R=100%DF=20%T=15%TG=15%S=20%A=20%F=30%RD=20%Q=20) T2(R=80%DF=20%T=15%TG=15%W=25%S=20%A=20%F=30%O=10%RD=20%Q =20) T3(R=80%DF=20%T=15%TG=15%W=25%S=20%A=20%F=30%O=10%RD=20%Q =20) T4(R=100%DF=20%T=15%TG=15%W=25%S=20%A=20%F=30%O=10%RD=20% Q=20) T5(R=100%DF=20%T=15%TG=15%W=25%S=20%A=20%F=30%O=10%RD=20% Q=20)
T6(R=100%DF=20%T=15%TG=15%W=25%S=20%A=20%F=30%O=10%RD=20% Q=20) T7(R=80%DF=20%T=15%TG=15%W=25%S=20%A=20%F=30%O=10%RD=20%Q =20) U1(R=50%DF=20%T=15%TG=15%TOS=0%IPL=100%UN=100%RIPL=100%RI D=100%RIPCK=100%RUCK=100%RUL=100%RUD=100) IE(R=50%DFI=40%T=15%TG=15%TOSI=0%CD=100%SI=100%DLI=100)
Once all of the reference fingerprints have been evaluated, Nmap orders them and prints the perfect matches (if there aren't too many). If there are no perfect matches, but some are very close, Nmap may print those. Guesses are more likely to be printed if the --osscan-guess option is given.
Dealing with Misidentified and Unidentified
Hosts
While Nmap has a huge database, it cannot detect everything. Nmap has no chance to detect most toasters, refrigerators, chairs, or automobiles
because they have no IP stack. Yet I wouldn't rule any of these out, given the ever-expanding list of connected devices. The Nmap fingerprint DB includes plenty of game consoles, phones, thermometers, cameras, interactive toys, and media players.
Having an IP address is necessary but not sufficient to guarantee a proper fingerprint. Nmap may still guess wrong or fail to produce any guess at all. Here are some suggestions for improving your results:
Upgrade to the latest Nmap
Many Linux distributions and other operating systems ship with ancient versions of Nmap. The Nmap OS database is improved with almost every release, so check your version number by running
nmap -V and then compare that to the latest available from
http://nmap.org/download.html. Installing the newest version takes only a few minutes on most platforms.
Scan all ports
When Nmap detects OS detection problems against a certain host, it will issue warnings. One of the most common is: “Warning: OS
1 open and 1 closed TCP port”. It is possible that such ports really are unavailable on the machine, but retrying your scan with -p- to scan all ports may find some that are responsive for OS detection. Doing a UDP scan (-sU) too can help even more, though it will slow the scan substantially.
Try a more aggressive guess
If Nmap says there are no matches close enough to print, something is probably wrong. Maybe a firewall or NAT box in the way is
modifying the probe or response packets. This can cause a hybrid situation where one group of tests look like they are from one OS, while another set look completely different. Adding the --osscan- guess may give more clues as to what is running.
Scan from a different location
The more network hops your packet has to go through to reach its target, the greater the chances that a network device will modify (or drop) the probe or response. NAT gateways, firewalls, and especially port forwarding can confuse OS detection. If you are scanning the IP of a load balancing device which simply redirects packets to a diverse network of servers, it isn't even clear what the “correct” OS detection result would be.
Many ISPs filter traffic to “bad” ports, and others use transparent proxies to redirect certain ports to their own servers. The port 25 or 80 you think are open on your target may actually be spoofed from your ISP to connect to ISP proxy servers. Another behavior which can confuse OS detection is when firewalls spoof TCP reset packets as if they are coming from the destination host. This is particularly common from port 113 (identd). Both the reset spoofing and transparent proxies can often be detected by noticing that every machine on a target network seems to exhibit the behavior—even those which otherwise seem to be down. If you detect any such nonsense, be sure to exclude these ports from your scan so they don't taint your results. You may also want to try from a completely different network location. The closer you are to the target, the more accurate the results will be. In a perfect case, you would always scan the target from the same network segment it resides on.