The Army and the Navy personnel are guarding the make-shift stage from where the Premiers will enjoy a panoramic view of the event.
Mr Abe and PM Modi landed by a special aircraft at the Babatpur airport in Varanasi around 4 pm. As the two left, the ministry of external affairs tweeted, "Taking Japan to the heart of India. In a special gesture PM invites PM @AbeShinzo to fly together to Varanasi."
Mr Abe, who is on a three-day day visit to India, will also meet a select group of people, both Indians and of people of Japanese origin, residing in Varanasi.
The two leaders will fly back to Delhi after a banquet, in which various dignitaries, including Uttar Pradesh Chief Minister Akhilesh Yadav, Governor Ram Naik and state ministers are likely to take part.
Track Latest News Live on NDTV.com and get news updates from India and around the world