The surname Fairfield certainly does not seem to be Irish so most likely descends from people from Britain that ended up in Ireland as a result of general migration or maybe as part of one of the plantations, it’s not usually possible to trace individuals or families involved in these unless they were landed gentry or military. The surname was quite rare in Ireland in the mid 1800s with two main areas - county Down and County Wicklow around Aughrim where you found your Fairfields. At the time of the 1901 census most of the people with the surname in counties Antrim & Down are shown as Church of Ireland, smaller number Presbyterian and Catholic which could fit with English and Scottish ancestry. At the same time it appears that all the Dublin & Wicklow Fairfields were Roman Catholic.
There are three Fairfield households that I can see in the Bray area on the 1901 census :
A widow Mary with her son and daughter living at Green Park Road in Bray, the son is a labourer and mother and daughter housekeepers
The second household is a William Fairfield and his wife Bridget living in Oldconnaught, Little Bray, technically County Dublin, but close to Bray - William is shown as a labourer..
The third Fairfield household is Michael & Catherine Fairfield also at Oldconnaught with their son & two daughters, the father & son are shown as ‘Gardener, D. Servant’ - all except one of the daughters were born Dublin or County Dublin. Although they lived on the other side of Bray, it’s possible that Michael & his son worked at Kilruddery..
I found one other Fairfield link to the area, a marriage in 1886 in the Catholic Church at Bray of a Margaret Fairfield to a Thomas Giles - both full age and living at Kilruddery, he is a labourer, she is a servant, their fathers are James Giles a labourer & David Fairfield a gardener both deceased.
Is this couple your Fairfield connection to Kilruddery ?
I can see this couple with a number of children on the 1901 census living at Newtownvevay near Bray - Thomas is shown as a Gardener, he and Margaret both born County Wicklow most of their children are shown as born County Dublin. Based on births of the children it seems that the family lived for a time in the area near Dalkey and later Blackrock county Dublin. There should be some further details of these families available on the free IrishGenealogy website, which has Civil Birth, Marriage an Death records up to the privacy cut-offs - ~100 years for births, 75 for marriages and 50 for deaths.
You can add details of your Fairfield line to the Ancestors database, that way any others researching the same family lines can find the details and possibly make contact - see the Ancestor Database