I imagine because most Summoner's are accustomed to soloing with the original cast time and so the slight gain of summoning an avatar faster wouldn't outweigh the more notable gain of physical acc/atk or magical acc/atk.
Well you've done legion and I haven't yet, is hitrate/magic hitrate that bad to consider putting acc/macc merits in? If all a player uses SMN for is lower end content it's not worth putting those merits in since you'll already be hitting the accuracy/magic hitrate cap.
5/5 magic attack is a given, but physical attack merits have been proven to be unnoticable, bordering on a total waste of points in group 1 compared to something else.