That's because no other ethnicity lives in that stretch of land hence why all of it was called Barbaria in the past. Shewa and western Hararghe however were more or less Ethiosemitic speaking strongholds.
Medieval geographers delineated Bilad al-Barabir’s and Barr as-Sumal’s extent via the coastlines (hence coastal Berbera to coastal Barawa) and the rivers (i.e the Nile of Mogadishu) but how far it stretched inland is not explicitly mentioned. No historic writer mentioned ‘Ethiosemitic’ in any historic chronicle, that is modern secondary interpretations of the texts such as those from al-Maqrizi. This is why we have recent Ethiopianist scholars substituting a reference to ‘they spoke Habash’ as being equal to ‘They spoke an Ethiosemitic language’.
We know Ifat conquered the kingdom of Shewa roughly two hundred years before the Age of the Futuh, that is enough time for a strong Somali influence to have taken root in that area by the time Ifat collapsed a century later and when Adal was born (It took the Oromos less time than that to replace large parts of the Horn).
In the end, however, it doesn’t really matter because any supposed reference to the Harla prior to the Futuh is not concrete, but interestingly enough the so-called ‘Karla’ or ‘Kazla’ are still mentioned as living east of the Abyssinians and north of the Zanj like the modern Somali, while the actual
concrete references in the Futuh only link them with the Somali people and no other modern group. If you add to this the fact that all of the ‘remnant’ Harla groups in the Horn spoke a Somali dialect and only traced themselves back ancestrally with a Somali genealogical table, and its pretty much an open and shut case.
At-least if it concerned any other group, but things get political when it comes to a group like the Somali people because they are an existential threat.
