"Pygmy populations" are recognized in several places over the world, especially in Western Africa and in Southeast Asia (Philippine "negritos," for instance). Broadly defined as "small-bodied Homo sapiens" (compared with neighboring populations), their origins and the nature of the processes involved in the maintenance of their phenotype over time are highly debated. Major results have been recently obtained from population genetics on present-day negrito populations, but their evolutionary history remains largely unresolved. We present and discuss the Upper Pleistocene human remains recovered from Tabon Cave and Callao Cave in the Philippines, which are potentially highly relevant to these research questions. Human fossils have been recovered in large numbers from Tabon Cave (Palawan Island) but mainly from reworked and mixed sediments from several archaeological layers. We review and synthesize the long and meticulous collaborative work done on the archives left from the 1960s excavations and on the field. The results demonstrate the long history of human occupations in the cave, since at least ~30,000 BP. The examination of the Tabon human remains shows a large variability: large and robust for one part of the sample, and small and gracile for the other part. The latter would fit quite comfortably within the range of variation of Philippine negritos. Farther north, on Luzon Island, the human third metatarsal recently recovered from Callao Cave and dated to ~66,000 BP is now the oldest direct evidence of human presence in the Philippines. Previous data show that, compared with H. sapiens (including Philippine negritos), this bone presents a very small size and several unusual morphological characteristics. We present a new analytical approach using three-dimensional geometric morphometrics for comparing the Callao fossil to a wide array of extant Asian mammals, including nonhuman primates and H. sapiens. The results demonstrate that the shape of the Callao metatarsal is definitely closer to humans than to any other groups. The fossil clearly belongs to the genus Homo; however, it remains at the margin of the variation range of H. sapiens. Because of its great antiquity and the presence of another diminutive species of the genus Homo in the Wallace area during this time period (H. floresiensis), we discuss here in detail the affinities and potential relatedness of the Callao fossil with negritos that are found today on Luzon Island.