Just saw a post on AH.com, mentioning a Salon article, that before Carrie Fisher signed on to become Princess Leia, Lucas was going for Jodie Foster to take the role. Foster refused, because she was already working on a Disney movie, and didn't want to break contract.
Salon is ludicrously unreliable. Glorified buzzfeed. So I looked up the source, and it's a Jodie Foster interview where she only says that they considered a younger Leia early on, and that she auditioned, but had to drop out (of the auditioning process) because of another role that was a sure thing.
She hadn't been offered the role.
The article also mention other actresses who were contenders for the role before that. Meryl Streep auditioned for the role(she was a theater actress back then), and in the running were also Jane Seymour, Kim Basinger, Geena Davis, Glenn Close, Jessica Lange, Sigourney Weaver, Anjelica Huston and Sissy Spacek(Amy Irving may also have been a contender).
So, what would change with any of those in the Princess Leia role? Many of those were either noticeably older or noticeably younger(especially Foster, which would have been around 13 back then) than Carrie Fisher, so either 'Luke and Leia are twins' is out, or there will have to be a different casting for Luke Skywalker(former being much more likely IMO).
As far as I know, there were auditions, yes. The recordings are even on YouTube. And of those named here, Amy Irving is actually one that we
know auditioned, just like Foster and Sissy Spacek. All the others mentioned are possible, but I've never seen test footage of that. There are some other actresses that auditioned, and of whom test footage can be found online. That would be Cindy Williams, Linda Purl, Karen Allen, Koo Stark and Terri Nunn. (Note that Karen Allen was later cast in
Raiders of the Lost Ark, and Koo Stark got a bit part in ANH, although that was relegated to the deleted scenes later.)
Meanwhile, actors William Katt, Robby Benson, Will Seltzer and Charles Martin Smith auditioned for the role of Luke. (Kurt Russell, too, but he was soon a contender for the role of Han instead.) I gather that William Katt was the other major candidate, next to Hamill.
The list of potential "Han Solos" is even weirder. Besides Kurt Russell, you had Nick Nolte, Burt Reynolds, Sylvester Stallone, Christopher Walken, James Caan, Perry King... and also Al Pacino (who famously said "I don't get what this is about" and left)... and then the really "out there" options: Jack Nicholson, Robert De Niro, Kelsey Grammer, Bill Murray, Steve Martin and Chevy Chase.
(Do note that
initially, Han Solo was going to be a green alien, and more of a comical character. So maybe that explains the latter candidates.)
As an aside: Lucas first offered the role of Tarkin to Christopher Lee, and initially wanted Peter Cushing for the role of Obi-Wan Kenobi. He also offered the latter part to Toshiro Mifune. (Who turned it down, and also didn't speak English.) Mel Blanc and Richard Dreyfuss were considered for the voice of C-3PO. And, of course, Lucas initially considered Orson Welles for the voice of Darth Vader.
Anyway, I'm assuming that the film develops similarly otherwise, so certain 'weirder' options still get ruled out by that. Han Solo won't be a quasi-comical alien sidekick. They nixed that early on. And they figured out that they needed Leia to be a bit older very early on in production, so that pretty much rules out Jodie Foster (14 at the time of filming) and Terri Nunn (15 at the time of filming). Likewise, they wanted Luke to be young and boyish, which would ultimately rule out Will Seltzer (even though Lucas liked his acting). This would in turn probably rule out Meryl Streep, Jessica Lange, Sigourney Weaver, Sissy Spacek, Glenn Close and Cindy Williams. Meanwhile, Koo Stark was ruled out because... well, frankly, she wasn't much of an actress. So I assume that would still be the case regardless.
As far as ages go, Mark Hamill was 5 years older than Carrie Fisher to begin with. So some age differences can be made to work. During filming, the age comparison of the most credible candidates would look like this:
Carrie Fisher: 19
Geena Davis: 19
Linda Purl: 21
Kim Basinger: 22
Amy Irving: 22
Karen Allen: 24
Jane Seymour: 24
Anjelica Huston: 24
Robby Benson: 20
Charles Martin Smith: 22
William Katt: 24
Mark Hamill: 24
Kurt Russell: 24
Perry King: 27
Sylvester Stallone: 30
Harrison Ford: 33
Christopher Walken: 33
Nick Nolte: 35
James Caan: 36
Burt Reynolds: 40