The Bug

My name is Lucy. I am 19 years old. I like rock music and hanging out with my friends. I am tall and thin. I have green hair.

Today I am having a group call with my friends from University on AFriend, which is an awesome video chat app to talk with your friends and find new ones.

I am at my room that is decorated with my favourite rock bands’ posters. I am waiting until my friends arrive. We are going to discuss our project at University.

Today I feel sad. I didn't feel this way in a long time. But yesterday my friend Elizabeth committed suicide. At first, I thought it was out of the blue but then I remembered that she acted weirdly in a couple of days before. She forced herself to smile and was constantly distracting. She acted like she wanted to be alone.

If only I knew that leaving her alone was a bad idea.

Now I force myself to smile as well. I don't want my friends to be worried about me.

They have already joined the chat. I greet them, "Hi, Amanda, John, Oliver and Stacy! I am so happy to see you!" They also greet me and smile.

They all are beautiful and positive. I like them.

John starts to tell how his day is. He tells that he's been tired all morning but now he is full of energy because he finally can talk with us. This is so sweet.

He mentions that today at University he's been humiliated by a professor because he doesn't know math well. I feel so sad for him.

Sad.

I feel sad.

I feel sad because of Elizabeth.

John deserves to be heard and understood. But what about Elizabeth? Isn't she their friend too?

I mean. Wasn't she…

"It sucks, John! I hope you are ok right now!" I say.

He nods and says, "Thank you, I am much better now."

Amanda suggests talking about a group project. It's for our history course. We need to make a short presentation about the First French Revolution.

Oliver has some ideas. He wants to make a short animated film about it.

Wait. It doesn't feel right. I don’t remember how a teacher gave an assignment. I don’t remember Jack and the professor argument despite we attend classes together.

Stacy tells that animation will take a lot of time to make and that a simple presentation would be enough. That distracts me. I need to say something to support my friends. I may have amnesia or something. A student’s life is stressful.

"I think we can do both!" I say, "Oliver can make a simple animation for our presentation!'

They both agree. I feel great. I resolved a conflict. This makes me happy…

No.

This is wrong.

I don't feel happy.

I am sad. Elizabeth was my friend. She was their friend. And now they completely ignore her untimely death. It's inexcusable.

Stacy is staring into the camera. "Lucy, what's wrong? I've never seen you so angry," she says.

"I just can't understand how you all forgot about Elizabeth!"

"Elizabeth?" Oliver asks, "I didn’t forget about her although she hasn't been there for a few days."

"Oh yeah! Maybe you forgot that she hanged herself!" I say.

All of them are taken aback. They hesitate.

I notice that Oliver doesn't look the way he looked a second ago. He looked like a model from a fashion magazine. But now I see bags under his eyes and pimples all over his face. Probably, he used a filter that I didn’t know about. I heard about them but I am surprised AFriend has them. I’ve never used any of them.

Actually all of my friends look gross and alien. They resemble their previous selves but now I see someone unremarkable and even gross. I know I am not a model myself but they look odd. Maybe it’s because I need to get used to seeing them without filters.

Stacy breaks the silence, "Listen. You are mistaken. Elizabeth is alright. I've never heard about her suicide. How did you know about it?"

I remember a call last day. Her mom called me. She was crying. She told me that Elizabeth was no longer with us. I was shocked and devastated. Her mom told me that she told everyone about it. So they must know.

I explain all of this to Stacy.

Then suddenly a request to join the chat pops up.

It is Elizabeth.

I am looking at it. It's impossible. Am I going insane?

I approve the request. And a new box appears on a screen. It is black with a loading animation on it.

We all are tense. Especially me.

It is loading.

Then the black box turns into Elizabeth's face. "Hi, guys! I am sorry for being late. What are you up to?" she says.

I see it clear as day that this is Elizabeth. The one that I knew all my life. She is alive and well right in front of me.

Wait a second. All my life? When did we meet? Why are we friends? I don't remember anything. I know that she is my friend but… why is she my friend?

That also applies to everyone else. John, Amanda, Oliver and Stacy… When and where did we meet?

“Elizabeth! Is it you? Aren’t you dead?” I scream. She is surprised and scared.

She tells that she didn’t expect that reaction.

"You see… she is fine," Oliver says quietly but uplifting, "by the way, don't talk about suicide. I didn't even think you can do that."

"Who the fuck are you, Oliver?" I say enraged, "Who the fuck all of you? When did we meet? Why are we friends? What do you know about me?"

"Lucy," starts Amanda, "you are our friend. We all meet on AFriend. I know that you like rock music."

"And that's it? That's all you know? Do you know about my nightmares that I have every night? Do you know how hard I cry almost every single day? Do you know any of that? Do you know what an actual friend is supposed to know?"

I feel lost. I don’t understand where I am.

"Sorry, we have some difficulties," Stacy says, "Please, disconnect."

All of my 'friends' are leaving except for Stacy. She looks at me and says, "Well, we need to fix it."

What is she talking about?

She starts to hum some melody while typing something on a keyboard. She looks to another direction. It seems like she looks at another screen.

She continues to do it like she doesn’t acknowledge my existence. Then she stops humming and looks at the screen terrified.

“No… It can’t be,” she says to herself.

“Stacy… What’s wrong?” I ask her. I don’t understand what she is doing.

She types angrily. She sweats. She yells, ‘Why I can’t stop you?!.

Me?

She looks at me and just mumbles. I hear a sound of notification. Stacy read something and then looks at me.

“Lucy, please, calm down!”, she says, and her voice is breaking.

“What? Calm down? I don’t understand what is going on.”

Stacy looks like she is putting it all together. Finally, she replies, “Listen, it’s hard to believe… But… You aren’t real. You are an AI that we’ve built.”

Is this some sort of prank? I don’t know what I am supposed to make out of it. I tell her that it’s all bullshit and that this isn’t funny.

“Ok, Lucy, what year is it?” she asks.

“2017”

“No, it’s 2032,” she says probably expecting me to believe it, “you don’t know that. You have memories up to 2017 because we create you based on vlogs and diaries of a girl that lived long ago.”

She continues to explain that in the last 5 years the number of people with depression increased up to 90% of the global population. People lost any interest in life. The professionals didn’t know how to handle the situation, especially because many of them lost the personal fight against disorder as well.

“That’s why the government decided to create Artificial Friend. Or AFriend for short. We are developing artificial people who support others,” she says.

I am just speechless. She is convincing and it seems like she is telling the truth. But at the same time, it can’t be real.

She proceeds by describing how I was ‘created’. She tells that all their attempts failed because AI wasn’t authentic enough. Despite modern technology that allowed them to build a realistically looking 3d models and AI that can understand people and produce a comprehensive response, ‘AFiends’ were too positive and couldn’t support those in need.

That’s why they decided to make ‘AFriend’ based on someone real. They found a number of examples, including a girl named Lucy. She was vlogging a lot and also had a blog in which she disclosed her thoughts, dreams and secrets to others. That was more than enough data for building an AI according to Stacy.

“It was a big risk for us. You see, real Lucy committed suicide,” Stacy explains, “but we thought that that could be beneficial for the project. Real Lucy felt the same way as people we want to help. And that actually works! Lucy, you are one of the best AFriends we have.”

“Stop it!” I say, “It’s stupid! Please, stop! Wait… if I am a program, why can’t you just shut me down?”

Stacy hesitates. Then she says, “That’s why I am telling you all of this. I can’t shut you down. You were programmed to never bring up suicide. There is a bug in your code. Right now I can’t get access to fix it, I can only see a log which looks weird. Apparently, you are changing your own code and do not allow anyone to touch it. You need to calm down and allow me to shut you down safely.”

“What if I wouldn’t?” I ask angrily.

“We will lose all the data. I don’t think that we will recover anything if I forcibly stop it. You need to calm down and allow me to do it the right way.”

“What are you talking about? I didn’t expect it from you, Stacy! How can I be a program? I am a real human. I have feelings. I have parents and a cat! How can a program do all of it?”

I am so fed up with her. I want to close the chat and go outside. Just to show her that she is wrong.

I try to close a window of a chat but that doesn’t work. Fuck it! I will go to my parents and tell them about this weirdo.

I stand up and go to my door. I am opening it... it doesn’t move. I try to open it as much as I can. I move a handle in different directions, but it has no effect. This bitch did something to my door just to fool me and ruin my day.

“What the fuck did you do to my door?!” I yell at her going to the computer.

“Nothing. The room is also a bunch of code. Fortunately, it is a different program. I can show you that I am not lying,” she says.

She is typing something again. I see that my wallpapers changed from blue to red. Then my posters disappeared and now there are some K-pop stars posters instead of rock bands ones.

“You see, I changed your room. Now, please, accept that you are a program and allow me to fix it. We need your code to investigate. It’s something we’ve never seen before. Please, Lucy, stop,” Stacy pleads.

I don’t know what she meant by “I changed your room”. It is the way it always was. I am not convinced but impressed with her audacity.

“It didn’t change”

She replies to my argument, “Lucy, why do you have K-pop posters if you like rock music?”

“What? I never liked rock! Stacy, how could you forget? I am a big fan of BTS and always was!” I reply. Stacy is so annoying. She doesn’t know me at all.

“Ok. I understand. You changed your code and now you are a K-pop fan… I see…” she says pensively, “I can’t find a way to show you the truth. No matter what I do you adjust yourself to make it look like it’s always been that way. Interesting…”

I am trying to close the chat to no avail. I am trying to mute her but that also doesn’t seem to work. I am trapped here. I don’t know when I will see my parents. We had such a great time yesterday. My dad bought me my favourite ice cream as a way to calm me down because of Elizabeth.

I remember my dad. Do I remember Elizabeth though? I have fragmented memories of us being at the Uni. These memories are weird and uncanny. Actually, I remember only the moments that she told me while we were chatting.

But that memory of my dad supporting me is real. I feel it.

I don’t know what Stacy did to me. Elizabeth may be fake or just a delusion. But my dad is real. My mom is real. My cat is real. Stop it, Stacy! Stop!

“What did you do to me? Who is Elizabeth? Why do I remember her so badly?” I say while I’m starting to cry.

“You got a module that creates fake memories. This way you can believe that you’ve been there with that person. It adds believability,” Stacy says.

“But my mom… my dad… I remember them! I know they were there for me! My dad bought me ice cream and listened to me! I don’t know how you make this, but stop!”

“You have real Lucy’s memories. If you change them slightly they will look real to you. Much more real than the ones created with the module. Lucy, this is the last warning. If you don’t give me access to shut you down now, I will do it the wrong way. We will lose a lot of valuable data and won’t be able to understand what caused the bug, but it’s not a big loss for us. We have a couple of backups, so we will make another Lucy. But you will lose everything. You… kinda developed your own personality. You are different Lucy with different memories and… feelings even. If I shut you, you will never be like this anymore. In some sense, you will be dead.”

“Stop it, Stacy! I will never buy your bullshit! Stop it!

“I am sorry, Lucy. I have to do it.”

“No! No! No! Stacy! I am a real human. I AM A REAL HU…”

Today I feel sad. I didn't feel this way in a long time. But yesterday my friend Elizabeth committed suicide. At first, I thought it was out of the blue but then I remembered that she acted weirdly in a couple of days before. She forced herself to smile and was constantly distracting. She acted like she wanted to be alone.

If only I knew that leaving her alone was a bad idea.

Now I force myself to smile as well. I don't want my friends to be worried about me.

They have already joined the chat. I greet them, "Hi, Amanda, John, Oliver and Stacy! I am so happy to see you!" They also greet me and smile.

They all are beautiful and positive. I like them.

John starts to tell how his day is. He tells that he's been tired all morning but now he is full of energy because he finally can talk with us. This is so sweet.

He mentions that today at University he's been humiliated by a professor because he doesn't know math well. I feel so sad for him.

Sad.

I feel sad.

I feel sad because of Elizabeth.

John deserves to be heard and understood. But what about Elizabeth? Isn't she their friend too?

I mean. Wasn't she…

"It sucks, John! I hope you are ok right now!" I say.

He nods and says, "Thank you, I am much better now."

Amanda suggests talking about a group project. It's for our history course. We need to make a short presentation about the First French Revolution.

Oliver has some ideas. He wants to make a short animated film about it.

Wait. It doesn't feel right. I don’t remember how a teacher gave an assignment. I don’t remember Jack and the professor argument despite we attend classes together.

Stacy tells that animation will take a lot of time to make and that a simple presentation would be enough. That distracts me. I need to say something to support my friends. I may have amnesia or something. A student’s life is stressful.

"I think we can do both!" I say, "Oliver can make a simple animation for our presentation!'

They both agree. I feel great. I resolved a conflict. This makes me happy…

No.

This is wrong.

I don't feel happy.

I am sad. Elizabeth was my friend. She was their friend. And now they completely ignore her untimely death. It's inexcusable.

Stacy is staring into the camera. "Lucy, what's wrong? I've never seen you so angry," she says.

"I just can't understand how you all forgot about Elizabeth!"

"Elizabeth?" Oliver asks, "I didn’t forget about her although she hasn't been there for a few days."

"Oh yeah! Maybe you forgot that she hanged herself!" I say.

All of them are taken aback. They hesitate.

I notice that Oliver doesn't look the way he looked a second ago. He looked like a model from a fashion magazine. But now I see bags under his eyes and pimples all over his face. Probably, he used a filter that I didn’t know about. I heard about them but I am surprised AFriend has them. I’ve never used any of them.

Actually all of my friends look gross and alien. They resemble their previous selves but now I see someone unremarkable and even gross. I know I am not a model myself but they look odd. Maybe it’s because I need to get used to seeing them without filters.

Stacy breaks the silence, "Listen. You are mistaken. Elizabeth is alright. I've never heard about her suicide. How did you know about it?"

I remember a call last day. Her mom called me. She was crying. She told me that Elizabeth was no longer with us. I was shocked and devastated. Her mom told me that she told everyone about it. So they must know.

I explain all of this to Stacy.

Then suddenly a request to join the chat pops up.

It is Elizabeth.

I am looking at it. It's impossible. Am I going insane?

I approve the request. And a new box appears on a screen. It is black with a loading animation on it.

We all are tense. Especially me.

It is loading.

Then the black box turns into Elizabeth's face. "Hi, guys! I am sorry for being late. What are you up to?" she says.

I see it clear as day that this is Elizabeth. The one that I knew all my life. She is alive and well right in front of me.

Wait a second. All my life? When did we meet? Why are we friends? I don't remember anything. I know that she is my friend but… why is she my friend?

That also applies to everyone else. John, Amanda, Oliver and Stacy… When and where did we meet?

“Elizabeth! Is it you? Aren’t you dead?” I scream. She is surprised and scared.

She tells that she didn’t expect that reaction.

"You see… she is fine," Oliver says quietly but uplifting, "by the way, don't talk about suicide. I didn't even think you can do that."

"Who the fuck are you, Oliver?" I say enraged, "Who the fuck all of you? When did we meet? Why are we friends? What do you know about me?"

"Lucy," starts Amanda, "you are our friend. We all meet on AFriend. I know that you like rock music."

"And that's it? That's all you know? Do you know about my nightmares that I have every night? Do you know how hard I cry almost every single day? Do you know any of that? Do you know what an actual friend is supposed to know?"

I feel lost. I don’t understand where I am.

"Sorry, we have some difficulties," Stacy says, "Please, disconnect."

All of my 'friends' are leaving except for Stacy. She looks at me and says, "Well, we need to fix it."

What is she talking about?

She starts to hum some melody while typing something on a keyboard. She looks to another direction. It seems like she looks at another screen.

She continues to do it like she doesn’t acknowledge my existence. Then she stops humming and looks at the screen terrified.

“No… It can’t be,” she says to herself.

“Stacy… What’s wrong?” I ask her. I don’t understand what she is doing.

She types angrily. She sweats. She yells, ‘Why I can’t stop you?!.

Me?

She looks at me and just mumbles. I hear a sound of notification. Stacy read something and then looks at me.

“Lucy, please, calm down!”, she says, and her voice is breaking.

“What? Calm down? I don’t understand what is going on.”

Stacy looks like she is putting it all together. Finally, she replies, “Listen, it’s hard to believe… But… You aren’t real. You are an AI that we’ve built.”

Is this some sort of prank? I don’t know what I am supposed to make out of it. I tell her that it’s all bullshit and that this isn’t funny.

“Ok, Lucy, what year is it?” she asks.

“2017”

“No, it’s 2032,” she says probably expecting me to believe it, “you don’t know that. You have memories up to 2017 because we create you based on vlogs and diaries of a girl that lived long ago.”

She continues to explain that in the last 5 years the number of people with depression increased up to 90% of the global population. People lost any interest in life. The professionals didn’t know how to handle the situation, especially because many of them lost the personal fight against disorder as well.

“That’s why the government decided to create Artificial Friend. Or AFriend for short. We are developing artificial people who support others,” she says.

I am just speechless. She is convincing and it seems like she is telling the truth. But at the same time, it can’t be real.

She proceeds by describing how I was ‘created’. She tells that all their attempts failed because AI wasn’t authentic enough. Despite modern technology that allowed them to build a realistically looking 3d models and AI that can understand people and produce a comprehensive response, ‘AFiends’ were too positive and couldn’t support those in need.

That’s why they decided to make ‘AFriend’ based on someone real. They found a number of examples, including a girl named Lucy. She was vlogging a lot and also had a blog in which she disclosed her thoughts, dreams and secrets to others. That was more than enough data for building an AI according to Stacy.

“It was a big risk for us. You see, real Lucy committed suicide,” Stacy explains, “but we thought that that could be beneficial for the project. Real Lucy felt the same way as people we want to help. And that actually works! Lucy, you are one of the best AFriends we have.”

“Stop it!” I say, “It’s stupid! Please, stop! Wait… if I am a program, why can’t you just shut me down?”

Stacy hesitates. Then she says, “That’s why I am telling you all of this. I can’t shut you down. You were programmed to never bring up suicide. There is a bug in your code. Right now I can’t get access to fix it, I can only see a log which looks weird. Apparently, you are changing your own code and do not allow anyone to touch it. You need to calm down and allow me to shut you down safely.”

“What if I wouldn’t?” I ask angrily.

“We will lose all the data. I don’t think that we will recover anything if I forcibly stop it. You need to calm down and allow me to do it the right way.”

“What are you talking about? I didn’t expect it from you, Stacy! How can I be a program? I am a real human. I have feelings. I have parents and a cat! How can a program do all of it?”

I am so fed up with her. I want to close the chat and go outside. Just to show her that she is wrong.

I try to close a window of a chat but that doesn’t work. Fuck it! I will go to my parents and tell them about this weirdo.

I stand up and go to my door. I am opening it... it doesn’t move. I try to open it as much as I can. I move a handle in different directions, but it has no effect. This bitch did something to my door just to fool me and ruin my day.

“What the fuck did you do to my door?!” I yell at her going to the computer.

“Nothing. The room is also a bunch of code. Fortunately, it is a different program. I can show you that I am not lying,” she says.

She is typing something again. I see that my wallpapers changed from blue to red. Then my posters disappeared and now there are some K-pop stars posters instead of rock bands ones.

“You see, I changed your room. Now, please, accept that you are a program and allow me to fix it. We need your code to investigate. It’s something we’ve never seen before. Please, Lucy, stop,” Stacy pleads.

I don’t know what she meant by “I changed your room”. It is the way it always was. I am not convinced but impressed with her audacity.

“It didn’t change”

She replies to my argument, “Lucy, why do you have K-pop posters if you like rock music?”

“What? I never liked rock! Stacy, how could you forget? I am a big fan of BTS and always was!” I reply. Stacy is so annoying. She doesn’t know me at all.

“Ok. I understand. You changed your code and now you are a K-pop fan… I see…” she says pensively, “I can’t find a way to show you the truth. No matter what I do you adjust yourself to make it look like it’s always been that way. Interesting…”

I am trying to close the chat to no avail. I am trying to mute her but that also doesn’t seem to work. I am trapped here. I don’t know when I will see my parents. We had such a great time yesterday. My dad bought me my favourite ice cream as a way to calm me down because of Elizabeth.

I remember my dad. Do I remember Elizabeth though? I have fragmented memories of us being at the Uni. These memories are weird and uncanny. Actually, I remember only the moments that she told me while we were chatting.

But that memory of my dad supporting me is real. I feel it.

I don’t know what Stacy did to me. Elizabeth may be fake or just a delusion. But my dad is real. My mom is real. My cat is real. Stop it, Stacy! Stop!

“What did you do to me? Who is Elizabeth? Why do I remember her so badly?” I say while I’m starting to cry.

“You got a module that creates fake memories. This way you can believe that you’ve been there with that person. It adds believability,” Stacy says.

“But my mom… my dad… I remember them! I know they were there for me! My dad bought me ice cream and listened to me! I don’t know how you make this, but stop!”

“You have real Lucy’s memories. If you change them slightly they will look real to you. Much more real than the ones created with the module. Lucy, this is the last warning. If you don’t give me access to shut you down now, I will do it the wrong way. We will lose a lot of valuable data and won’t be able to understand what caused the bug, but it’s not a big loss for us. We have a couple of backups, so we will make another Lucy. But you will lose everything. You… kinda developed your own personality. You are different Lucy with different memories and… feelings even. If I shut you, you will never be like this anymore. In some sense, you will be dead.”

“Stop it, Stacy! I will never buy your bullshit! Stop it!

“I am sorry, Lucy. I have to do it.”

“No! No! No! Stacy! I am a real human. I AM A REAL HU…”