Besides user count, the number of federated instances, posts, and comments will also increase server costs. Its possible that federating from many instances has a larger performance penalty than having a high user count.