I'm also experiencing this 8GB of RAM problem, although the "-winxp" is working as a fix for now. I also have problems with ArmA, unless I limit my ram (using MSConfig).
System...
Windows Vista Ultimate x64 + SP2
Intel Q9300
8GB DDR II (G.Skill)
ATI 1GB HD4890 (HIS), CCC v9.6, 1920x1200
Playing with the ArmA Demo version 1.02.