Absolutely. Here are the big (20 volume) Oxford English Dictionary
1. a.1.a One who lies (i.e. sleeps or lodges) out, i.e. in the open air, or away from a place with which he is connected by business or otherwise.
1676 D'Urfey Mad. Fickle ii. i. (1677) 11 Out-liers, comers, and goers. 1705 Stanhope Paraphr. III. 201 He dispatches another Message to the Highways and Hedges, to fetch in all the Outlyers. a 1742 Bentley Lett. 59 (R.) The party‥sent messengers to all their outliers within twenty miles of Cambridge to come to their election. 1866 N. & Q. 19 May 421/1 Outliers are soldiers (generally married men) who, when there is not sufficient barrack accommodation, receive an allowance‥and provide themselves with lodgings.
b.1.b One that lies outside the pale, an outsider.
1690 D'Urfey Collin's Walk A vij b, Every worthy and true English Protestant of the Establish'd Church (for I have no hopes of the Outlyers). 1826 Lamb Lett., to Bernard Barton 147, I do not know how friends will relish it, but we outlyers, honorary friends, like it very well.
c.1.c An animal that lies outside the house, fold, or park; esp. an outlying deer.
a 1658 Cleveland Gen. Poems etc. (1677) 157 It is but Trifling sport for you to pull down an Out-lyer, unless you leap the Pale and let slip at the Herd. 1892 Ainslie Land of Burns 37 (E.D.D.) It wauken'd burdies frae the bough, An' outlyers frae their lair. 1939 Joyce Finnegans Wake 97 From his holt‥the outlier, a white noelan‥, led bayers the run. 1976 Abingdon Herald 25 Nov. 7/2 Another outlier found near Sparsholt went through Sparsholt Copse, across to the Spinneys again, and back to Westcot. 1977 Field 13 Jan. 52/1 Hounds found an outlier at the back of Alexton village and hunted him past the hall.
2. a.2.a ‘A stone not taken from a quarry, but lying out in the field in a detached state' (Jam.); a boulder. Also †outlair. Sc.
1610 Burgh Rec. Aberdeen (Spalding Club) II. 300 The keaping stane to be of outlairis, frie wark, and boulted with irne. 1807 J. Hall Trav. Scot. II. 333 There is, in the parish of Ordiquhill, a large outlier of lime stone some tons weight, and no lime-rock to be found near it. 1846 Wright Ess. Mid. Ages II. xvii. 210 On a black moor called Monstone Edge, is a huge moor-stone or outlier. 1955 A. Thom in Jrnl. R. Statistical Soc. A. CXVIII. 275 Many of the circles have one or more outliers, i.e. single upright stones outside the ring. Ibid. 283 Why not also include Little Meg as an outlier to the circle at Long Meg and her Daughters?
b.2.b Geol. A portion or mass of a geological formation lying in situ at a distance from the main body to which it originally belonged, the intervening part having been removed by denudation.
1833 Lyell Princ. Geol. III. Gloss. 76 When a portion of a stratum occurs at some distance detached from the general mass‥some practical mineral surveyors call it an outlier, and the term is adopted in geological language. 1854 H. Miller Sch. & Schm. viii. (1857) 160 There lies in the Firth beyond, an outlier of the Lias. 1889 J. Croll Stellar Evolution 55 Occasional outliers of conglomerate on the Highland side of the fault.
c.2.c generally. An outlying portion or member of anything, detached from the main mass, body, or system to which it belongs. Also attrib.
1849 Ruskin Sev. Lamps ii. 54 Interrupted‥by great mountain outliers, isolated or branching from the central chain. 1854 R. G. Latham Races of Russia 39 Outlyers from the neighbouring Government of Esthonia. 1881 G. Allen Vignettes fr. Natures, Fall of the Year, Australia remains an isolated outlier of Asia to the present day. 1885 R. F. Burton tr. Arabian Nights' Entertainments V. 177 They took leave of him and departing to the outliers of the city, flew‥to their several abodes. 1926 [see Hattic a.]. 1928 Library Assoc. Rec. Dec. 244 The Central Library has wisely recruited several of the larger public libraries‥to act as outlier libraries. 1961 T. Landau Encycl. Librarianship (ed. 2) 259/2 Its ‘Outlier' libraries, which lend their specialized books and periodicals on the N.C.L.'s request when other resources fail, now number 281. 1973 Computers & Humanities VII. 136 What‥differentiates ‘La Comtesse d'Escarbagnas', at the top of the diagram, from the remaining plays? What is special about an outlier such as ‘Dom Garcie'? 1977 Jrnl. R. Soc. Arts CXXV. 269/2 The Library is an ‘outlier' library of the British Library.
3.3 Fishing. A set-line, out-line. U.S.
Add: [2.] d.2.d Statistics. An observation whose value lies outside the set of values considered likely according to some hypothesis (usu. one based on other observations); an isolated point. Also transf.
1907 Biometrika V. 312 The ‘exceptionals'‥are mostly ‘outliers' in the tables of pairs of distributions considered. Ibid. XXVIII. 312 (in figure) Outliers in samples of 10 & 15. Ibid. 317 Even when there are obvious outliers, the process may never get started at all. 1950 Ann. Math. Statistics XXI. 38 A natural statistic to use for testing an ‘outlier' is the difference between such an extreme observation and the sample mean. 1960 Technometrics Feb. 1 At the 1959 meetings of the American Statistical Association held in Washington D.C., Messrs. F.J. Anscombe and C. Daniel presented papers on the detection and rejection of ‘outliers', that is, observations thought to be maverick or unusual. 1973 Computers & Humanities VII. 136 What‥differentiates ‘La Comtesse d'Escarbagnas', at the top of the diagram, from the remaining plays? What is special about an outlier such as ‘Dom Garcie'? 1987 K. A. Rubinson Chem. Anal. iv. 132 The validity of outliers can be tested by using statistical methods."
The CD-ROM version of the OED is an excellent working tool and Oxford have just brought out a 2-volume historical thesaurus.
An aside on the actual word outliers from today's International Herald Tribune:
"The executives of the other companies asserted Tuesday that they believed BP was an outlier, cutting corners to save time and money in ways that they would not tolerate."
So it can have negative connotations!