You don’t say what denomination your family were but if they were RC your problem is that hardly any RC parishes in Co. Donegal have records earlier than around 1850. That makes it very difficult to trace them.
Usual advice in this situation is to exhaust all sources outside Ireland. Have you found the family in the 1851 Scottish census? Any information there? Check naturalisation papers, military records, obituaries, wills and gravestones in the hope that they mention a parish or townland in Ireland.
One possibility is to check whether the family claimed under the Poor Law in Scotland. If they did the files often record where they originated in Ireland, sometimes with next of kin. The Poor Law records for the Glasgow area are in the Mitchell Library. They will do looks ups for a small fee.
http://www.glasgowlife.org.uk/libraries/the-mitchell-library/Pages/home.aspx