If you already see the tidy little mounds indicating entryway into ant tunnels, your first step must be to eliminate as many of the pests and their homes as you can. A common, safe and environmentally efficient way to kill ants is to pour boiling water into the holes. The water will kill the ants you see and many you don't farther down in the tunnel. Full-strength vinegar is also an effective ant killer and has the added benefit of lasting a bit longer than plain water.
A variety of pesticide products are specifically designed for outdoor use in ant control. The products, applied as a liquid or gel or used as a solid bait, are designed to be transported by scout ants back to the nest to kill the queen. Liquids are particularly effective on pavers because they're most able to soak through cracks and reach beneath the stone. Test a small area before spraying the whole walk, as some products will permanently stain stone.
Once you've done your best to eliminate your current crop of ants, discourage new colonies from taking up residence with some preventive maintenance. First, clear the crevices of weeds and soil. You can pull the weeds, use environmentally safe products like vinegar, or use commercial weed-killers like Roundup.
After removing the weeds, fill the spaces between stone with a mix of paver sand and a granular insecticide to further keep ants at bay. Also use the insecticide as perimeter barrier.
Another option used by contractors is to add a polymer to the sand. This helps seal the cracks and make it more difficult for ants and weeds to burrow through the narrow slots.