The magical protection his mother gave him would cease to exist. As long as he was able to call that place home, Voldemort would never find him there. As long as he returned there for at least a little while each summer, he would be safe.
He would be James Potter (the arrogant, bullying side) all over again. Lets face it, Sirius is not exactly a good role model. Snape would finally have a legitimate reason to hate him. (;<
he would be seen as a murderer's friend, and thus needed to live an obsurced life from usual life.(Although he will live better off there than the Dursleys as he will be treated as a son to Sirius.)
Death Eaters یا Voldemort would have been able to find him and kill him. The shield his mother used to protect him would have vanished O__o the Dursley's may suck (and be fat) but at least they protected him for 16 years :/