Some balloons go up. What is the air composition in the balloon?
For a ballon to rise in air the mass of the contained gases + the weight of the ballon must be less than the air it displaces. So the ballon could be filled with hydrogen, helium, or hot air. For a party balloon, it is probably got helium in it. At atmospheric pressure, pure nitrogen would be only slightly less dense than air so the boyancy effect would be so weak that it would be highly unlikely that a party-size balloon with nitrogen in it would float in air. Also, because the gas in a party balloon is compressed a little (thus made more dense) because of the rubberyness of the ballon, this might more than offset any slight boyant force obtained from filling with pure nitrogen.