I do understand your point of view, but I respectfully disagree.

Precisely, I disagree with ethics not being dependent on data. Assuming that it does not depend, that means, no matter what, there is an ethical course of action. While this might be true, is this useful?

Keep in mind that this is not an excuse to be unethical, far from it. Instead, I do believe that, for any given situation, there are more than one ethical course of action, each with their respective pros and cons while still being ethical. Which one you chose is based on the situation, therefore, the data.

Picture a village that has a supply of potatoes and a shortage of carrots. You, as the mayor, can buy more potatoes or more carrots (or both). Either path will uphold the ethical ground that you must not let people starve, thus, you need to regularly buy food. Yet, the sensible decision here is to focus on carrots. If you buy potatoes, you risk spoiling food and you risk malnutrition for lack of variety, so buying potatoes can be viewed as less ethical or less correct than carrots.

The point, however, is that, if an algorithm is tasked with this decision, it must be fed the amount of each vegetable to compute how many of each it has to buy. Given more potatoes than carrots, buy carrots. This works, this is ethical, and this can be tampered with to make the algorithm buy the wrong item regardless of the actual situation.

Finally, you might argue that the ethical path here is to buy both always. This way there is always potatoes and carrots, no one will starve and your decision is not based on data. However, (1) why is there an algorithm then? (2) you might still spoil the excess potatoes, (3) you might not buy enough carrots, (4) you won’t be using your funds ideally, which might incur in not having enough for other supplies (e.g., pills) and, thus, be unethical.