In the past, I have tried to assign IP addresses and set up a true server and clients but without a true server package, the best you can do is peer-to-peer.
If you are running Windows 2000 Server then disregard anything I am about to say. If not, this may help.
The preferred setup for multiple machines to connect to the Internet is through a Gateway as show here:
Having said that, I have yet to set up my network that way because it requires two NICs in the gateway machine and also leaves it outside the protection of the hardware firewall of the router.
Without a dedicated Server and Gateway, you will be required to set up a peer-to-peer network. Fortunately this is simple using WinXP.
The first thing I would do is update all SP1 machines to SP2 as SP1 will soon, if not already, not be supported. Select one WinXP machine to start with, right-click on My Network Places, and select Properties.
Double-click on Network Setup Wizard and follow the steps until you get to the window sub-titled "Select a connection method." Select "Other".
You will now be at a window sub-titled "Other Internet connection methods..."
The example looks like this:
Continue with the setup and it will offer to create a setup disk for use with other machines. I created one as I have Win98 machines on my network. Since you have non-WinXP I would recommend the same. Use the floppy to start the setup for all your other machines. Do not hard code any IP addresses, they will be assigned dynamically.
This is the way that has worked for me so far. I have a gateway machine built and ready to add to the network but have not yet had the time to do so. Until then, my network is set up as I have described here.
The only drawback to setting up your network without a gateway is that each and every computer on the network must now have its own, dedicated protection suite of software; anti-virus, spyware blockers, etc. That is not as big a problem as it sounds as there are many good, free options available for each. Also, file and print sharing are not recomended. I use it but make sure that all shared files are read only and only store non-critical files there. If one computer needs information from another I can pull it from the host. Nothing, however can be placed in a Read Only folder from another computer.
We have some really good computer techs on this forum. Hopefully one of them will jump in and give more information ... if not better advice. Until then, I wish you the best and hope this helps your problem. Please let us know how it works out.