I had a look for this family in the 1901 & 1911 Irish censuses but without success. Likewise I can’t find the parent’s marriage. There are a couple of Kavanagh-Kavanagh marriages but not the right forenames or occupation.
I noticed this young William fostered out. I wondered if his mother had died, and this had led to him being placed with another family. Just supposition:
http://www.census.nationalarchives.ie/pages/1911/Dublin/New_Kilmainham/St__Mary_s_Terrace/11124/
If William K senior was a serving police officer in Ireland, then they are usually in the censuses under their initials only. A search under WK & male brings up 199 possibles. Since we don't know where he was born or his age it makes it impossible to narrow that down. They are often listed by their former occupation or the euphemism “farmer’s son”, as in this WK example, who was RC, 36 and from Sligo but serving in Belfast (he was single so it’s not the right one, just an example):
http://www.census.nationalarchives.ie/pages/1911/Antrim/Dock_Ward/Glenravel_Street/136718/
Other thing to know about the Royal Irish Constabulary (RIC) is that they normally never posted you to a county where you or your wife had family connections. So if born in Co Dublin he wouldn’t serve there. If he was serving there, then he was born in another county. In addition to the RIC, there was the Dublin Metropolitan Police Force. He might have served with them. (Not sure what their recruitment policies were). First step may be to check the RIC and DMP records to see if they had a William Kavanagh who married Julia Kavanagh. In the RIC, you had to have served 7 years before you could marry and marriage details, including the wife’s county of birth, are nearly always recorded on the service records, because of the need to avoid a posting to the wife’s home county.
Was William perhaps serving with some other police force in Scotland or England? Perhaps his wife came home to give birth to William junior but otherwise the family lived in Britain. Again, as examples, huge numbers of officers in the London Metropolitan police and City of Glasgow police forces were from Ireland. In the late 1880s 20% of the City of Glasgow force were Irish born and the Met had similar figures. So, if you can't find the family in Ireland, search English & Scottish censuses and marriage records.