DOWN at Brisbane's Riverbend bookshop, anticipation is building. Where is Lemony Snicket? Is he late? Is he hiding? Does he in fact exist at all?

The gathered children and their guardians scan the crowd for the mysterious author whose A Series of Unfortunate Events has been in The New York Times bestseller list for more than a year, and looks destined for big-time popularity.

Suddenly a plumpish man materialises from the footpath. He is wearing a sombre black suit, white shirt and a tie, rather like a cross between a Victorian undertaker and the third Blues Brother minus shades and hat.

He bears the bad news that Mr Snicket will not be coming, due to an unfortunate event at a picnic that day.

The children giggle, some out of nervousness, and Daniel Handler, 31, admonishes them slowly and deliberately for laughing at Snicket's misfortune. The crowd falls quiet. The author, he says, has been bitten by a dangerous bug.

Then Handler falls face down to the floor in a mock convulsion. By now the penny has dropped that the man lying prone on the podium platform, one arm twitching to comic effect, is the mysterious Lemony Snicket himself.

It's quite a performance; the kids love him and you get the feeling Handler genuinely enjoys playing up to the kids. By the end of the evening he has sung a woeful (in both senses of the word) ballad while playing an accordion, accused the bookshop's owner, Susie Wilson, of trying to steal his jacket ("I know there's not much money in running a bookshop"), and read a passage from his book where the Baudelaires cook puttanesca sauce.

And he talked, between sips of pale milkless tea, to every child who presented their book to be signed. Wilson, despite being labelled a thief, is ecstatic: They've never had an author quite so entertaining or engaging before. If only all book readings could be like this.

Since writing his first book two years ago (The Bad Beginning, about the wretched fortunes of the three Baudelaire children after the incineration of their parents), Handler has had plenty of opportunity to evolve his "act" and the mysterious persona of Lemony Snicket, the narrator of the dark tales.

"I met another children's author who said what she liked to do was erase the mystery behind children's books, behind books in general and writing. And I thought, why would you want to do that? The mystery behind books is somebody sitting at a table making something up.

"It's not a very good story. It's not like a mystery behind (he pauses here while he fishes for an example) glass, where you get to go to a factory or something.

"So I tried to think of something that would increase, rather than decrease the mystery." Shadowy photographs of the author taken from the back, and perplexing dedications in his books (Ask him who Beatrice is, everyone urges), have indeed mystified readers as to his identity and increased interest. But more recently the charade has been dropped a little and Handler has emerged from behind his character. The February edition of glossy celebrity bible Vanity Fair named Handler as the current "in" Gen-X bard – which is ironic considering his critique of the fickleness of fashion and "in-ness" in his sixth book The Ersatz Elevator.

The "out" bard, incidentally, was Dave Eggers, author of A Breathtaking Work of Staggering Genius. Apparently the New York media loves pitting the two, possibly because Eggers also goes to great lengths to create an aura of mystery around his appearances.

Being a literary celebrity, however, has its down side. Handler has recently had to delist his phone number after being called up and visited by obsessive fans of his adult books, which are very dark (one he calls "an incest comedy").

"I've met people who are obsessed with my adult novels and that's sort of disturbing, but when there's a 10-year-old who has read your novel 12 times and wants to talk to you about it, that's very charming.

"So if you're going to be a literary celebrity it's nice to have it be with children who are enthusiastic and not kooky."

Handler, who now lives in New York with his wife, grew up in San Francisco, the son of an accountant and college dean. In 1993, he won an Olin fellowship which enabled him to write his first novel The Basic Eight. Then for two years he wrote comedy for a radio show.

The seven Unfortunate Events books were born out of a pitch to a children's book editor at HarperCollins who thought his idea of a "mock Gothic novel" was brilliant. Any doubts about the wisdom of such a bleak series – the book was initially a sleeper – were soon put to rest.

Handler's helpers are keen to tout Snicket's books as the new Harry Potter.

Comparisons are inevitable, with some readers finding the orphans Violet, Klaus and Sunny Baudelaire less to their taste than the boy wizard and his friends. But Handler points out that his young heroes are not meant to seem real and are based on stock characters in the Gothic genre.

"It's interesting for children to read characters that they can project themselves wholeheartedly into. So that's one of the reasons why we don't know so much about them."

This playfulness with literary conventions is likely to go over the head of most of his younger audience but Handler does not think the references to classic books and myths are wasted.

The literary sensibility and sophistication that pervades his writing, Handler says, comes simply from having always read a lot.

"I grew up in a household where everyone was reading and books were valued. There were always lots of good books in the house and I was allowed to read anything I wanted."

The darkness of his stories – in settings, tone and subject matter – derives from his insatiable appetite for stories "where creepy things were happening in dark, creepy castles".

"I don't remember a time when I wasn't reading Edgar Allan Poe," he says.

Among other injustices, in A Series of Unfortunate Events, children are imprisoned in cages, dangled out of buildings and routinely betrayed by those adults entrusted to care for them. Behind the stories lurks the idea that despite your best efforts or intrinsic goodness, you aren't always going to be rewarded.

"I don't think children have to be taught that, they already know it. You're very, very young the first time you see some small act of injustice, whether it is someone awarded something they didn't deserve, or perhaps you as a child have done something wrong and not been punished and you begin to see that the world does not go the way you're taught it should.

"The books I found annoying when I was young were full of bullies who turned out to be nice people or mean teachers who got their comeuppance and all these things I didn't see happening around me.

"So often in the books there's a sense that the children's plans to get them out of trouble are in some way overcome by the evil plottings of adults, and I think children feel that way all the time. Children will discuss what they want to do or what they feel ought to happen next and then it doesn't happen at all." Given the wry, black humour of his stories, described once as "anarchy masked as propriety", Handler is surprised that they have been so well received.

"I'm flabbergasted (by their success). When I first started writing these books I thought they would be noble failures," he says.