Location of Victoria Peak
The terms Victoria Peak and "The Peak" are actually quite loosely applied depending upon the situation. From the point of view of official government defined political districts The Peak actually doesn't exist, it is an area which covers parts of Central, Wanchai and Southern District.
From a physical point of view The Peak covers the entire raised area in the center of Hong Kong island, from the Lung Fu Shan Country Park in the west all the way along the "Dragons Back" as far as the east of the island.
However when people say "The Peak" they are actually more likely to be talking about the residential area from around the Peak Tower, and extending South West to Mount Kellett and East as far as the area around Magazine Gap.
This is actually quite a large area and is well populated with residential buildings which were of course originally inhabited by the British Colonials, and are now home to the more wealthy of Hong Kong residents of all nationalities.
For the point of view of tourists and visitors "The Peak" is an even smaller area, basically that immediately around the high terminus of the Peak Tram, in the Peak Tower and nearby Peak Galleria.