According to the rootsireland site, Robert and Jane married at Ballyclug Church of Ireland on 30.7.1856. (Not Ballymena 1st Presbyterian church). Robert was 22, a stonecutter and a bachelor. Robert’s address then was Airdrie (a town outside Glasgow) so he had evidently been living and working there. His father was James Bell, a blacksmith. Jane was 26, a publican, and a widow. Her previous married name was Kell. She lived in Harryville (a suburb of Ballymena). Her father was James Johnson, a farmer.
Tradition was to marry in the bride's church, after which she’d normally attend her husband. So that accounts for a marriage being in one church and the children’s baptisms in another. Ballyclug Church of Ireland was probably Jane’s church and could be where she was baptised. Her earlier marriage was in the same church. So that tends to confirm it was her family’s church.
Ballymena 1st Presbyterian was evidently the Bell family church, if the children were baptised there. So if you are looking for Robert Magill Bell’s baptism, I would search those records. Ballymena 1st’s baptisms start in 1812 and there’s a copy in PRONI in Belfast (MIC1P/114).
Statutory birth registration started in Ireland in 1864 so you can get birth certificates for the children born after that. But for those prior to it, the church baptism records are the only records of their births.
Andrew was born in Larne St, Ballykeel (part of Ballymena) in 1865. David Service Bell and John Bell were both born at James St, Ballymena. Robert Bell died in Ballymena Workhouse, though he had probably only gone there because he was ill. The workhouse admission records should be in PRONI and may tell you when he was admitted, and what his normal address was. His death certificate doesn’t record his normal address. Jane died in Henry St, so that was probably where the family were living at that time.
Ballymena Workhouse is still there. After the workhouse closed, the hospital wing became a state run hospital and today is the Waveney Hospital. Part of the old workhouse building has been restored and there’s a commemorative plaque on the wall, as you go in, for visitors to see.
Jane Johnston’s 1st marriage was on 19.4.1851 in Ballymena to John Kell. It was also in Ballyclug Church of Ireland. Jane was a spinster, aged 21, her father James was an Innkeeper. Her husband was John Kell, a weaver, from Cross townland, aged 22, bachelor. His father was William a farmer.
I see this gravestone in Ballymarlow Church of Ireland graveyard. It appears to relate to your family:
Erected by Jane Bell of Harryville in memory of her beloved relatives and husband Robert Bell who died 19th December 1884 aged 52 years The above Jane Bell fell asleep in Jesus 4th August 1889 aged 59 years Also their beloved daughter Lizzie Mary McCullough called home 25th April 1890 aged 31 years Also their beloved son Robert James Bell who was 'absent from the body present with the Lord' November 22nd 1936 aged 74 years 'Him that cometh to me I will in no wise cast out ' John 6. 37 Also their loving daughter in law Charlotte T Bell, aged 81 years died 21st Nov 1932 and her infant son Talbot died 19th May 1889 Also their beloved Grandchild Anna I. Bell aged 43 years died 27th January 1933
Talbot Bell was born 19.4.1888 in Mill St, Ballymena to Robert James Bell and Charlotte McDowell.
https://civilrecords.irishgenealogy.ie/churchrecords/images/birth_returns/births_1888/02519/1933406.pdf
They had married on 1.12.1884 in Ballymena Register Office. Robert’s father was Robert Bell stonecutter.
https://civilrecords.irishgenealogy.ie/churchrecords/images/birth_returns/births_1888/02519/1933406.pdf
Robert & Charlotte in 1901:
http://www.census.nationalarchives.ie/pages/1901/Antrim/Kells/Ballee/930828/
And in 1911:
http://www.census.nationalarchives.ie/pages/1911/Down/Pottinger__part_of_/Knock_Road/229650/
Hope this helps.