Don't forget that your TOTAL UPLOAD BANDWIDTH is "shared" by all who are on your server at the same time, doing whatever they are doing. For example, the more downloads going on at the same time will lower the individual file download speeds as your upload bandwidth is used up. You have only so much upload bandwidth which must be shared with everyone that's on your server.
The more UPLOAD BANDWIDTH (not download bandwidth) you have the better when running a server.