Steve,
Joseph’s age in 1886 (22) suggests he was born around 1864. Statutory birth registration started in Ireland on 1.1.1864. I searched for Joseph Reilly/Riley births in 1864 & 5 in Co Armagh. I only found one that might be for Armagh. It was registered in Newry which covers both part of Armagh & part of Co Down. Joseph Reilly b 7.2.1864 mother’s maiden name McCoy. I don’t know the father’s first name. That’s not shown in the indexes. It should be free to view on the irishgenealogy site but it’s not there (their records have gaps). However it is on the GRONI website. You would need to pay to view it though, using the “search registrations” option:
https://geni.nidirect.gov.uk
You will need to open an account and buy some credits. It costs £2.50 (sterling) to a view a certificate.
If Joseph was born before 1.1.1864, you would need to search church baptism records. To do that I would need to know his precise denomination in order to advise where to search.
Having said that, I searched the on-line trees on Ancestry and found just one for this family (owner Stephen Hardistry who I suspect may be you). That gives Joseph’s father as William John Reilly and his mother as Ann McQuade (married in Loughgall, Co Armagh 13.2. 1856). It also gives details of a sibling William born 1866 in Armagh.
Unfortunately I cannot find a birth that matches that. There are however births for George Reilly 26.5.1866 in Clonmacash (to those parents) plus twins Henry & Jacob on 2.12.1868. Here’s George:
https://civilrecords.irishgenealogy.ie/churchrecords/images/birth_returns/births_1866/03532/2299811.pdf
Henry & Jacob are again missing from the irishgenealogy site but their births can be found on the GRONI site. There may have been more births post 1868, I didn’t check.
Given that the parents married in 1856, it’s possible that Joseph & William were born pre 1864. That would explain the absence of birth certificates. People in Ireland in the 1800s didn’t celebrate birthdays and often didn’t really know their age. They are often just guesses. Here’s the 1856 marriage:
https://civilrecords.irishgenealogy.ie/churchrecords/images/marriage_returns/marriages_1856/09501/5443388.pdf
The marriage was in Loughgall Church of Ireland. Tradition was to marry in the bride’s church, so that suggests she at least was Anglican. William’s townland was Fernagreevagh and Ann’s looks to be Derrycrew (both in the parish of Loughgall).
If the family was Church of Ireland, then Loughgall’s baptisms start in 1706 (with some gaps). I am not sure if they are on-line anywhere but there is a copy in PRONI in Belfast. (Personal visit required to view them). If you are unable to go yourself, you could employ a researcher. Researchers in the PRONI area: http://sgni.net