From the readings, there seemed to be a few main causes of the radiation overdoses delivered by the Therac-25. The investigation documents posted by Virginia tech described the race conditions in the code that were perhaps the fatal flaw in the code. But there were also several other things that seemed to be a flagrant disregard for safety. For one, the code was written in assembly instead of C. Maybe it wasn’t as common to use C back in the 1980’s (although I would assume it was common to use C since it was invented in the 70’s), but anyone with any experience with assembly can tell you how much easier it is to write buggy code in assembly as opposed to C. Additionally, the system relied solely on software for safety checks and didn’t include the hardware backups that were present in earlier iterations of the machine. The Hackday articles also said that “the software appeared to have been written by a programmer with little experience coding for real-time systems” and that “Timing analysis wasn’t performed. Unit testing never happened. Fault trees for both hardware and software were not created.”. These are important things to do if you are building a software for a piece of hardware than can deliver lethal dosages of radiation. Also, the article notes that this important coding task was given to only one programmer. A safety critical piece of software should be checked over and understood by as many people as humanly possible to root out bugs.
I think the whole project points to people who refused to take responsibility for their system’s safety. Anyone designing hardware should have had second thoughts about taking fail safes out of a design like that. Additionally, the lack of testing for the software is disturbing. Unit testing is a standard for the most benign of programs. It should be a requirement for any safety critical system. Although I know little about real time systems, it sounds like timing analysis and fault trees are an industry standard that was also flaunted.
I also suspect the programmer may have had too high an estimation of his skill level. If you are building something that can affect people’s lives in such a dramatic fashion, it is important to know when to say you’re out of your league. The programmer seemed to know enough to build a system that worked most of the time, but if this is your first time implementing real time or safety critical software, you should know your limitations enough to look for outside help from someone with more experience or expertise.
Projects like these should be approached with the mindset that you will spend 1% of time in building the system and 99% of your time validating. When bugs can be fatal, tests must be exhaustive and thorough. Although it may not be the most economical option, it is a moral requirement. Engineers developing systems like these should be thoroughly trained by others with more experience, as well as being aware of industry standards and best practices for risk mitigation.
The best article that I think I’ve read on the gender gap in STEM was “Why the STEM gender gap is overblown”. The article is more well thought out than the somewhat inflammatory title would imply. It shows that in most STEM field (excluding computer science) women aren’t really underrepresented. I also appreciate that it fights against the idea that women should be required to go into STEM field. While these communities should be open and welcoming to everyone who is interested, women shouldn’t feel ashamed to go into female dominated industries or do whatever interests them. But I think the face that almost all STEM fields don’t have women underrepresented highlights the humongous problem that the computer industry has. We aren’t fostering an open and inviting environment where women who are interested in computers feel like they can belong. The industry is heavily male dominated. Of the internships I’ve worked at, I would say the average female to male ratio I encountered was about 1 to 20. The column from quartz entitled “I had so many advantages, and I barely made it: Pinterest engineer of Silicon Valley sexism” describes how challenging this can be if you’re a woman in the field. It’s just a difficult environment, which surely discourages lots of potential talent. Because this is such an ingrained problem I think the onus is mostly on the computing industry to make sure workplaces and schools to develop environments that are welcoming to both men and women.
When it comes to diversity, I like the quote “To quantify diversity by race or skin color is lazy and disrespectful. I’m not asking nor arguing we have less diversity. No, instead I challenge us to widen our scope and view diversity from an additional vantage point”. The Bloomberg article on recruiting from Howard University I think highlights this point well. Silicon Valley companies tend to go to high prestige University’s like MIT or Stanford for their recruitment. In some ways this makes sense for them as the universities have spent significant resources screening out bad candidates, so there is a very high probability whoever you hire from these universities will be a hard working, smart, effective employee. For reasons that I won’t discuss in this blog, these schools tend to pick students from middle to upper socioeconomic brackets, which are dominated by whites and asians. Companies have an economic incentive to hire from these institutions and so their demographics look similar to the colleges. One way to fix the problem would be to go further down the food chain and address the root problems as to why top schools don’t accept many students from lower socio-economic classes (who don’t tend to be white or asian as often). Businesses don’t really care about skin color, they care about making money, so they will continue to hire from places they know produce reliable workers. But while the short term saved recruiting costs of these practices make sense, companies gain to benefit from recruiting from colleges (like Howard) that have different admission criteria than Harvard or Princeton. While you might have to spend a little more time sorting out workers you don’t want, you stand to gain a diversity of thought that can greatly enhance your problem solving abilities. Tech would gain a more diverse workforce if they reached out to recruit at less prestigious schools and would also simultaneously open the opportunity that tech provides to people of all backgrounds. Having a workforce diverse in thought also tends to make it more inviting to people of different backgrounds, so I suspect this might go a long way to helping the gender gap in tech as well.
I think that companies are ethically obligated to support a good work life balance. An important tenet of Catholic Social Teaching is that “the economy exists for the person, not the person for the economy”. When companies refuse to respect the work life balance of their employees exist to serve the company and ensure it makes money. They should be approaching business as a means for their employees to sustain their physical well being so the can continue to serve God and man. When employers don’t respect their employees time it shows a disordered understanding of the function of work. I think the article title “I Came to San Francisco to Change My Life: I Found a Tribe of Depressed Workaholics Living on Top of One Another” perhaps best sums up the idea of work coming before people. It’s a significant ethical problem when companies are creating a tribe of depressed workaholics instead of enabling people to grow into better versions of themselves.
Work life balance has definitely played into my choice of careers. One of the first things I do when I evaluate whether I want to work at a company or not it is to find someone who has worked at the company before and ask them how many hours they expected to work per week. If it’s much over forty, I won’t apply. A employer that doesn’t consider work life balance signals to me that it isn’t a company that cares much about its employees, and therefor isn’t a company I would be interested in working at.
Work life balance is one of the many reasons I refuse to consider working in silicon valley. While this is certainly a general statement that doesn’t apply to all companies, many tech companies in the area don’t provide a good work life balance. For instance, one of the articles posted for us to read talks about a developer who went to the Google I/O 2016 Conference, where the writer says “I saw something horrifying, something I couldn’t shake from the moment I saw it… Eat. Sleep. Code. Repeat. was printed on everything.” This certainly doesn’t sound like work life balance. Google understands work life balance too. I found an article with an abstract that reads “Google research shows that those who rigidly separate their personal and work lives are significantly happier about their well-being than those who tend to blur the lines between the two”. Yet the silicon valley culture is to do as much as possible to blur the lines and keep employees at work, with lavish workplace perks like nap pods and personal chefs. It’s mixing off time and work, when research (and I think most people’s intuitions) signals this is a very bad idea. I find it very difficult to justify working 70 hours per week when I have other people I care about outside of work and other parts of my life to balance. Plus, I would have almost no time to enjoy all the money that I presumably would be making for working that many hours.
Our code of ethics has four main parts. They are do no harm, respect of system users, inclusivity, and responsible use of automation and AI. We chose these categories because when we listed our specifics, they seemed to be able to fit everything we had listed. We tried to include specific guidelines that are focused on human rather than technical problems.
There are definitely weaknesses in our document. Specifics fail to include every ethical decisions you can encounter, and generalities are broad enough to interpret in many different ways. This issue can only be addressed by carefully thinking through decisions in the spirit of the document, looking at how generalities arise from specific instances for guidance. We probably missed a lot of specifics, and we tried to avoid overarching generalities in hopes that readers can build generalities from our specific instructions of how to act. I don’t think there is really a way to address this flaw which is inherit in all codes of ethics.
A code of ethics is really only useful in that it forces what you value into your conscious mind. That way you can rationally consider your ethics, think how you would act in certain situations, and modify if need be. I don’t think codes of ethics do a particularly good job as guidance for individual situations. The circumstances often make only the most general part of a code of ethics applicable to a problem.
I believe that moving from job to job is somewhat inevitable in the current day in age. Forbes writes that “staying employed at the same company for over two years on average is going to make you earn less over your lifetime by about 50% or more.” While money is certainly not the only factor I will consider when evaluating a job, 50% less earning potential is enough to give me second thought about staying put for as long as I can at a single company. Also, in the tech industry, it is important to stay up to date on your learning about the latest advances in the field. I can see myself being stuck in a job and feeling like my skill set is becoming obsolete. This could become a liability for me in the event that I lose my job. Moving jobs often is the easiest way to gain new knowledge and make sure that I continue to be competitive in the workplace.
I believe the idea of employees coming and going at a quickened pace will quickly become more accepted (if it isn’t already, CNN has an article titled The new normal: 4 job changes by the time you’re 32) and expected by companies. I don’t have a moral issue with this type of behavior. The contract between me and my employer is that I will work to the best of my ability for them. In return, they pay me. I’ll do my best to uphold my end of the bargain for every company I work for. If I find a company that can assuage my concerns about not switching jobs (mainly keeping an up to date skill set and competitive pay) I would have no problem staying with them. Employment is an agreement that both sides enter understanding either can leave when they no long feel it’s mutually beneficial. Companies can fire employees at any time and employees should in turn be able to walk away at any time. In the past, there might have been an implicit contract that companies will take care of you if you don’t walk away. Loyalty was a part of the contract for both sides. Companies provided pensions and the means to work your way up the ladder. Because incentives were in place to stay at a company for a long time, employees did. Now companies no longer provide those incentives (pensions don’t exist and I found an article titled “The New Way to Climb the Career Ladder (Hint: You Don’t)” and so employees aren’t incentivized to stay. The idea of employment with and without loyalty are both legitimate types of contracts, the problem is just when employers and employees are on different pages about the type of contract they’re in and it seems someone has been wronged. If you’re job hopping in the first kind of contract, there may be a morality issue. But in an economy that doesn’t include loyalty in the contract, there is no real problem to be had in my mind.
There are a multitude things that I make up the ethos of the computing industry. I would like to focus on the two that I have the strongest feelings about, innovation and meritocracy. Sadly, I am disenchanted in many ways with the computing ethos, but especially these to tenets. That’s not to say it’s everything about it is wrong, but I think that the computer industry needs to re-think its core beliefs.
I believe innovation might be the most strongly cemented guiding principal the computer industry has. You don’t have to look far for evidence of this. For instance, the Guardian writes that one of the reasons that some in silicon valley are embracing universal basic income is that it “subsidizes disruptive technologies”. Disruptive technologies is just a buzz word for innovation. Mark Zuckerberg’s letter to investors just before the IPO of Facebook talks about their culture which he calls the hacker way and says that “in reality, hacking just means building something quickly or testing the boundaries of what can be done”. Sound like innovation to me. Why does the computing industry have such an obsession with innovation? They claim benevolent reasons. In the case of Facebook, it’s “strengthening how people relate to each other”. Often times, innovation is also linked to economic prosperity. Look at Silicon Valley and the vast amount of wealth it creates. While innovation is part of what makes computer science fun for me, I don’t think it is a worthy candidate to be a fundamental value. Innovation brings about new technologies. In my view, technology is like bullhorn for human behavior. It increases our power, to do both good and bad. Facebook has in fact allowed people from all over the globe to relate to each other, but people also stream murders on Facebook and bully kids to the point of suicide on Facebook. Innovation certainly increases wealth, but it definitely in an economic panacea that people make it out to be. Business Insider states in the article Silicon Valley’s prosperity paradox eplains how 76, 000 millionaires and billionaires fail to fix local poverty that 30% of silicon valley residents rely on public or private assistance . John Komos, an economics professor, states that “innovation is not the answer. Innovation is not going to give us nirvana or a just society or a good economy”. I think innovation makes tech fun to work in, but I think the tech community to often consider innovation a worthy pursuit. I don’t think it’s fair to say that because tech companies are innovative, they are contributing anything more to society than an insurance company or a restaurant. To be fair, innovation has contributed to more comfortable living for people as a whole (consider 1900 vs. 2000), but it’s a bad idea to consider innovation as solely a positive.
Meritocracy is a second tenet of the computing industry. Computers are widely available and part of our daily lives, so the idea goes that anyone with determination can become a skilled and successful computer programmer. This seems like a positive, we prefer people who work hard and are skilled at their craft to get ahead. But Quartz writes that ” Silicon Valley hums on the Soylent-libertarianism of a cabal of rich, white guys”. The fact that there is a single race and gender that dominate the largest tech city in the world would suggest there is little meritocracy in tech. The discussion of why this is might be best saved for another time, but it again the idea tech often has that they have the way of thinking to lead us into the future.
My main dislike of the computing ethos is the baked in assumption that the emphasis of innovation and meritocracy are making the world a better place for people. But the evidence I discussed earlier shows that innovation isn’t a panacea for the world’s problems. It’s really people using technology in a positive way that makes a difference. Computer science isn’t a meritocracy specifically because people assume that technology itself is enough to improve the world. Why would you bother take the time to teach programming to those least likely to access it or build an inclusive business culture when Facebook is already out there for the non tech initiated to do that?
It is important to study ethics in the context of computer science and engineering because computers permeate our live on such a daily basis. Many American workers check their smartphones on the way into work, spend their days parked behind computers at their desk. They then drive home where they have the option to use their laptop to order a pizza while watching Netflix. After calling it a night, they set the alarm on their phone in the morning. We use computers all day long for communication, work, and for relaxation. Because computers touch so many aspects of people’s lives, the designers of computer systems have an obligation to consider the ethics of the systems they build.
Bruce Schneier, a well known computer security expert, writes in his article titled ‘Click Here to Kill Everyone’ about the looming dangers of the internet of things, the next step in further integrating computers into our daily lives. He states that ‘we need to reverse the trend to connect everything to the internet’ because it poses such a large security risk to the world. Whether you agree with his argument and viewpoint or not, this is a prime example of why we should study the ethics of computer science. We have the option to do a lot of things in computer science. But just because we can do something doesn’t mean that we should do something. It is important to stop and consider whether or not the system we are building is ethical.
It is also important to consider the ethics of building a system once we have decided to move forward and build it. Very few people would argue that programming software for a airplane is considered unethical. But there are still many ways that this can be done in an unethical manner. Could you hire an un-vetted worker from halfway across the world to write your code for you? Sure. Could you write the code, give it a once over and say it looks good. Sure. But it definitely isn’t ethical to write safety critical software without a very thorough testing process or to outsource the work to someone who is unqualified to write the code. A mistake in this code could cost the lives of an entire aircraft of people. That’s why the ACM has the ethical guideline to avoid harm to others.
Another important aspect of ethical action in computer systems is how humans are allowed to interact with each other. Computers are often used as a medium of communication. Computer system designers have a ability (although limited in many ways) to control what is communicated on their systems. Is the internet considered a harbor of free speech where anyone can say anything without censor? Or do platforms have a responsibility to prevent offensive language? Or how about extending that a step further to illegal activity? Because computers are used in a social manner all the time, it is important to study the ethics of what system designers allow to be communicated on their products.
My name is Brent Marin, I am from Denver, CO and I study computer science. I chose computer science as my major because I wasn’t sure what I wanted to do. I figured computer science kept the most doors open. Everyone uses computers, so if I turned out to not like my major, my plan was to find a computer job at a company that did something else I was interested in and to slowly change over to a different career path. Luckily, I ended up loving computer science! I also enjoy running and playing piano in my spare time. From this class, I hope to think about ethical decisions I may be asked to make in the workplace so I have rehearsed how I am going to handle myself. I also hope to reflect on how we can play a part in making sure computers are changing communication for the better.
I think one of the more pressing ethical issues for computer scientists is privacy in regards to data collection. With recent advances in AI/Machine Learning, personal data is becoming more valuable than ever before. I think we need to stop and reflect on how we are going to handle this data. What is allowable to collect? What is allowable use of the data? This is a particularly interesting subject that I am interested in learning about. In a more general thread, I’m also interested in discussing AI and making sure that it has a positive impact as it becomes more widely rolled out.