As you progress through the story you will unlock more locations. These locations are unlocked by completing Arcana Release quests of the respective bosses.
This page is a work-in-progress.
Crystal Pillar in area 8 (not available right off the bat, but becomes available as you progress through the game)
Ahbor Desert does not have a Crystal Pillar. Unlocked by defeating Agni's Arcana Release (Agni Invades)
Crystal Pillar at area # 5 (*fun fact: this is the earliest pillar you can encounter); unlocked by defeating Takemikazuchi's Arcana Release (Takemikazuchi!)
Crystal Pillar Location Area 4; unlocked by defeating Bahamut's Arcana Release (Bahamut Attacks)
Crystal Pillar in Area 7 (in the swamp water by the walls); unlocked by defeating Azdaja's Arcana Release (Twin-Head Snake)
Crystal Pillar in area 8 (right next to a wall near a warp point); unlocked by defeating Grendel's Arcana Release (Sword Master)
The Scarlet Tower has no Crystal Pillar; unlocked by defeating Vermillion, the Blood Lord's Arcana Release (Red Conqueror)
Unlocked by defeating Hecatoncheir's Arcana Release (A-Maze-Ing Shot)
Unlocked by defeating Wild Nidhogg's Arcana Release (Dragon of Arcana)