Shoutcast or streamcast to save bandwidth

[darkpeace 0]

Shoutcast or Streamcast to save bandwidth thus even a 256kbps connection (upstream that is at the server end) could host ALOT more players, as everyone gets the 1 x 256 kbps stream outbound, and each client uploads whatever they need to to the server.

Combined with having clients Shoutcast some information to other clients, to save server bandwidth.

In this day and age, many people can afford 256 Kbps outgoing (a reasonable speed), with 1536 Kbps incoming (heaps)

And even if it was optimized to the point where 33.6 Kbps was playable (using the old system) thats only 30 players (max) that you would want per 10 Mbps link (and optimizing netcode this much is unlikely)

However with the above system, Shoutcast 1 stream from server to all clients, at 192-256 Kbps, which should be ample bandwidth for all players on the shoutcast, esp when each client can also shoutcast to other clients.

Thus if there is one idiot with a lame connection (playing in wrong country for example) he will only be affecting himself (unless he is driver/commander/gunner of a vehicle that is).

Anyone with a intermediate knowledge of networking concepts should be able to grasp the above.

Regards,

DarkPeace

ACT, Australia (where 1536/256 ADSL is all we can get)

[kegetys 2]

Dunno if you're talking about the same thing, but the shoutcast and streamcast I know about are made for streaming audio over an IP network, I doubt any game could have much use for such system (Except maybe for voice communication but that would propably be too complex to use with very little gain).

[darkpeace 0]

It would have huge savings for both voice and netdata. It may even help with cheaters.

[Raw 0]

The various methods for broadcasting data are nice, but they don't work well, since many ISP's block them and much of the network don't support them.

Check any p2p-forum, it is a constant source of discussion there, since a working broadcast protocol would be the holy grail of file sharing.

[darkpeace 0]

Laymans terms

If you want to send voice to 32 people at 16 kbps

It can use 32x16 Kbps (512 Kbps) OR

It can be shoutcasted and only use 16 Kbps

There is no reason why the same concept can be used for netcode, esp if clients can shoutcast to each other.

Have the game use ports 230x - 240x if need be (what ISP blocks game ports ? - I'll avoid them)

An ISP isn't going to block ports if they are used for gameplay and not pirating music, games, applications, etc

[kegetys 2]

There is no reason why the same concept can be used for netcode, esp if clients can shoutcast to each other.

Except that streaming audio (or any other media) and a regular ofp-like game netcode have very different goals; When you are streaming something, the first and most important thing is bandwidth. Latency can be pretty much ignored there, as it doesnt matter if a listener hears your stream 5 seconds later than when you broadcast it. Something like that would be a disaster in a "real time" game (having a 5000ms ping), where latency is the first and most important thing and you want as short path as possible from client to another, and shoutcast relies on different "relays" that transfer the stream forward and each step adds latency. Not to mention that shoutcast isn't intended for any game use, only for streaming audio, and it works only one way from the streamer to the listener.

Peer-to-peer based networking is a whole different deal though, and such could potentially help balance both bandwidth and cpu load from a central server machine to multiple peers only (Though this would also require much higher upstream bandwidth from each client than a client-server based approach, something which often isn't available). The problem is, that such thing would be quite hard to make, especially in a game like OFP where there are many non-player controlled units and vehicles all of which would somehow need to be balanced between each client, and still somehow manage to everyone to be aware of everything happening in the game in case a client drops out.

[Raw 0]

Laymans terms

If you want to send voice to 32 people at 16 kbps

It can use 32x16 Kbps (512 Kbps) OR

It can be shoutcasted and only use 16 Kbps

There is no reason why the same concept can be used for netcode, esp if clients can shoutcast to each other.

The thing is that the packets leave the server once, then each pice of network equipment along the line has to understand that there are multiple recipients, and preserve that information as needed or split the packets to several destinations if needed. It is not a question of blocked ports, it is a question of the abilities of the technology in use today.

The internet today is not ready for this, there is too much old equipment around. Besides, the ISP's are scared of this, since a single user could start streaming video and other bandwidth intensive stuff all over the world, which would generate huge amounts of traffic once the packets starts to split for different destinations. As I said earlier, this would be the holy grail of file sharing, since a single user could easily serve any number of download as if they were one, breaking the need for an overall p2p network up/down ratio of 1:1.

There may be that free lunch on the horizon, but it is not here yet.

Quote[/b] ]An ISP isn't going to block ports if they are used for gameplay and not pirating music, games, applications, etc

They don't give a damn anyway, you can usually use your connection for anything. The only thing they really care about is excessive bandwidth usage. This is why they are scared of broadcast protocols. A single user can easily generate far more traffic than his connection would otherwise limit.

[darkpeace 0]

I never said ISPs would like it, but it is easier to code than people might like to think it is.

It would be nice to have a huge Conquer the Island using this technology on a LAN with upwards of 384 players at a LAN duking it out on a huge battlefield. (similar to MFCTI 1.16+)

However if shoutcasts where limited to a given ISP, etc, then it would not cost the ISP in external bandwidth.

However 1 single Flashpoint 2 server with 384 players, would obviously be using more bandwidth than some low end Counter-Strike server, and it may even be more cost effective to host 384 players on 1 FP2 server, vs 384 players over 20 or so Counter-Strike servers with the forementioned shoutcasting technique.

Even coding both shoutcast and 'standard' netcode would not take much longer, and given the environment the server admin could select which was desired.

(Best of both worlds scenario)

[darkpeace 0]

Seam to be alot of people with a 'can't do' attitude around personally I would like to see it tried (flashpoint or otherwise, lan or internet) and results, checked, improved, etc

I mean ultimately it is the future of gaming, as even a low end link (upstream, kept within a given ISP) would be far more cost effective than the current 'each player needs truely dedicated bandwidth'.

And if people with 256 Kbps upstream can host, then the ISP saves on servers, and can keep datastreams internal, esp if specific port ranges are used that do not conflict with anything else.

And if Flashpoint 2 became more popular than CounterStrike I am sure that all of us would be very happy, esp if we don't all need to go hire $200/mo servers to play it in MultiPlayer.

(Remember Australia, 1536/256 ADSL is the best we can get, without forking out ALOT more $$$)

[WhoCares 0]

If I understand the concept of this technology correct, it is only usful in the distribution of identical information - send a packet once and add a header with a distrtibution list. Now the question is, are there sufficient identical packets - if OFP(2) always has individual information in each packet it sends to a client, then you are going to win nothing.

[darkpeace 0]

Correct me if I am wrong, but all players need to know the positions, movements, firing angles, etc of all other players ?

Now over 384 players on a single server, that is a shitload of indentical packets, no ?

One way is to only get sent data from players that are nearest you, this works, however if there are say 16 large battles going on at once, then 16 streamcasts (for each given area, the code for this is fairly basic) would be better than 384 dedicated 'channels' for game/MP data. It would:

1 - require less bandwidth

2 - be more CPU effient for the server, than processing what data to sent to 384 different players, based on what all players are doing (huge CPU usage)

If the server could intelligently calculate that instead of 1 shoutcast (send all players all data on all other players), that in fact 16 lower bandwidth shoutcasts (send all players all data on players near them, collectively, and a sideshout at lower bandwidth for stuff that is happening miles away, which player-2-player shoutcast bandwidth of say even 33.6kbps is suffient, and the server doesn't even 'need' to know about it, but could receive it anyway for verification/anti-cheat/etc) would be better. In fact it could, or rather, should dynamically setup 'shoutcasts' within a given port range based on what is going on.

Scenario 1 -

All players within 2 km sq, 1 shoutcast should do

Scenario 2 -

Players split over 4 battles, 4 shoutcasts, 1 sideacast.

Scenario 3 -

Players split over 192 micro battles, 32-192 thin bandwidth shoutcasts, minimal sidecasts.

(and so on, with everything in between)

In the above example I use sidecast to indicate data that is player-2-player on a thin upload channel (as it is faster than going from player-2-server-2-player ping wise, and even 33.6 Kbps could sustain it, thus players that normally lag, would lag less so, in perception to other players that is)

The server, capable of dynamically assigning X many shoutcasts based on conditions of battle, area, etc, all the variables, and a graph could be created, indicating to 'auto-optimize' the number of streams, esp if the game uses a sequential port range, even if only 64 UDP ports), as 192 Microbattles is very unlikely and overheads would prob make it pointless to use more than 64 shoutcasts at once.

There are going to be people that don't quite grasp the concept immediately, however with todays insanely fast CPUs and memory subsystems, and the internets limited bandwidth, esp to consumers in the upload direction, I do believe that ultimately the about would be the option, but not for 32 players on a decent link, however there is a point where (and I think it is about 96 players) the above becomes far more feasible than the current backward thinking method. And forward thinkers understand the above is only a starting point.

Personally I hope a codemaster with TCP networking and C++ skills picks up on the above.

In fact the above is almost comparable to the memory system in multi CPU Sun systems (to a degree, but surely some of you see the similarities), as Sun 'solved' the memory bottleneck (tends to appear in 6 CPU or greater systems) in a manner 'similar' to the above.

The above system might not be used in Flashpoint 2 (2005), but maybe in Flashpoint 3 (2007 - hopes)

Anyway it solves 2 problems:

1 - Flashpoint makes more effective use of bandwidth/netcode

2 - Flashpoint is economical to run, thus becoming more popular than CounterStrike by mid 2006 (for example)

And thanks to MMX/SSE,3Dnow!,etc it is more desirable to process such data as netcode in groups, vs per player/connection.

[suma 8]

Quote[/b] ]Correct me if I am wrong,

As you wish:

Quote[/b] ]but all players need to know the positions, movements, firing angles, etc of all other players ?

Not all players need to know everything about all players. Each players needs to know especialy about things that are near him, while things happening far from him can be sent very infrequently. If you should be receiving all state information about all players in the map, even your incoming connection would be saturated quickly - and it does not matter if you use multicast or not. The problem with multicast is it does not scale well - it works well when there are a few players close to each other, but once you will get saturated and you are unable to send everything to everyone, you are pretty much lost.

Moreover, when you track your players as a separate channels, it is possible to implement delta-compression (note: this is something which is not implemented in OPF, mostly because of original DirectPlay limitations when handling non-guaranteed messages). This is impossible when using multicast.

[darkpeace 0]

True, however, instead of designing it for 64 Kbps, you could cram 4x the data into a 256 Kbps stream, Delta compression would still work for the above anyway, just less so, it is only a design concept (much like Doom IPX MP is compared to current titles) and could be improved upon greatly.

So Flashpoint 2, will be closer to your typical FPS in Netcode ?

[darkpeace 0]

I was hoping that Flashpoint 2 would handle 384 Players on a LAN, there is a point when the above is far more effective than the current method of netcode that all current titles use, I could therefore deduce that FP2 will only support 64 - 80 players in Multiplayer ?

[darkpeace 0]

You can still have multiple streams as above anyway, instead of 1 x 10 Mbps stream, would would want 40 x 256 Kbps streams for example, it is a hybrid of multicast and the current method. (See above)

[suma 8]

Quote[/b] ]True, however, instead of designing it for 64 Kbps, you could cram 4x the data into a 256 Kbps stream

OFP1 was not hardcoded for 64 Kbps. Unfortunatelly there was a problem where the CPU load was growing very quickly with a growing number of players, therefore the bandwidth available was not used because server CPU was not able to create and send the data quickly enough - and this seems currently to be the main limit with OFP servers.

[darkpeace 0]

I can safely assume another workaround, has already been found to the high CPU / memory subsystem usage in FP1 though ? (3000 soldiers, insane amounts of units, etc really hits any system)

I hope FP2 server will get a boost from hyperthreading (above 55% CPU in task manager) and esp from quad or eight way servers.

So how are you guys solving the high CPU usage issue in FP2 anyway ? (figured that the more crap you can process in blocks, the better, esp going beyond standard x86/x87 and into the added instruction sets for multimedia and even the newer 64bit systems)

Still, 384 players on a LAN, any chances ?

[Raw 0]

Darkpeace, you are still not getting the point. It is not a matter of if the ISPs will like it or not, it is a matter of if their equipment will support it. The broadcast protocols are not a purely clientside solution, they rely heavily on features which must be present in the network equipment to work.

At the moment, the support for this on the internet is very limited at best. With no incentive for the ISPs to provide support for these protocols, it is not likely to happen soon.

Look at some file sharing development forums. This question pops up all the time on them and it always ends the same way: "It would be great but at the moment it is not possible.". These are guys who, unlike darkpeace, know what they are talking about. Trust me, if it was possible, every file sharing app would use it, since bandwidth management and preservation is by far the single largest problem facing them.

Perhaps it will be possible in 3-5 years, but it is not now and it will require significant investments in the internet infrastructure to make it possible. BI could put all the best developers on the task and it would still not be possible since the network is not ready for it. It would be lika having a phone but no line to connect it to.

Grow up. The fact thar something would be good does not make it possible.

[SpecOp9 0]

This technical talk is making me go haywire...

The only thing sticking out was 384 people on a LAN.

Could'nt Flashpoint 2 use a similar method to something like Star Wars Galaxies? or Planet Side?

I know the servers for those games are gigantic power monster machines... But would it even be nearly possible?

[darkpeace 0]

Might as well kill this thread off, lazy ass ISPs holding us back.

Finding 384 players that want to LAN Flashpoint is near impossible, the highest I have heard of is 64, and the server could handle more (human) players, so long as AI was kept to a minimum.

What is the maximum number of players a server can host anway ?, I mean setting 3072 players is all good and well, but at 64 players will it just reject connections ? (Flashpoint1)

Just that over the net finding 384 players is easy enough

Guess I'll just have to wait....

BTW: If you get a chance to use VBS

VESL Training Server|54ms|131.236.20.224:2302

VESL Match Server|61ms|131.236.20.58:2302

A30VESLNET00061.VESL.local|53ms|131.236.20.138:2302

All Australian servers, all using ADFA/ADFA2 mod

[hellfish6 7]

VESL Training Server|54ms|131.236.20.224:2302

VESL Match Server|61ms|131.236.20.58:2302

A30VESLNET00061.VESL.local|53ms|131.236.20.138:2302

All Australian servers, all using ADFA/ADFA2 mod

How can anyone get a chance to use VBS? It's only for the military and government.

[darkpeace 0]

By chance, you might just happen to work for, study at, see or experience VBS1. For example if you live in Australia and study at the Australian Defence Force Academy (ADFA).

You know....by...err....'chance' (wink wink, nudge nudge)

[shinRaiden 0]

Back on topic here...

Quote[/b] ]it is possible to implement delta-compression (note: this is something which is not implemented in OPF, mostly because of original DirectPlay limitations when handling non-guaranteed messages)

Hmm... shall we theorize that this has been addressed in the DirectX modernization with all the other goodies?

Quote[/b] ]Unfortunatelly there was a problem where the CPU load was growing very quickly with a growing number of players, therefore the bandwidth available was not used because server CPU was not able to create and send the data quickly enough

So the 64kb was an artificial throttle to control CPU loadage? That could be good news to us 56k'ers.

Argh. Devs on the forum this close to E3, it is unhealthy to my sanity.