Policy at scale: What makes an actionable complaint?

24 March 2020

Here’s something you may not know about abuse desks: they’re busy. The further up the “food chain” you get, the smaller the relative number of complaints you’ll see, but almost any abuse desk will receive at least several thousand complaints per year.

Dealing with complaints, then, is somewhat art and somewhat science. There are questions that need to be resolved and issues that need to be handled. This post will discuss the questions to be resolved when considering whether a complaint is even actionable.

Questions to be resolved

When a complaint is received, the first and most obvious question that must be resolved is, “Is this actionable?” We must recognize that not every complaint specifies a violation of policy and does not contain enough detail to make it actionable.

No violation of policy

Read morePolicy at scale: Understanding the issue

Here’s an illustration from my very first job in policy enforcement: One day we received a spam complaint about a customer that they were sending mail that the person had not opted in to receive. Here was the content of the message that they were complaining about:

Thank you for submitting your resume to us regarding the (name of position) opening. Unfortunately, we have decided to move in a different direction with filling this position. We will keep your resume on file and will be happy to let you know if a different position for which you may be better suited arises in the future.

Paraphrase of an email that was complained about

I’ve often said that had this email had different content (“Congratulations! We would like to offer you a position with our firm paying $250,000 per year plus benefits and options.”) we never would have seen that complaint. And, “content that the recipient doesn’t appreciate” will rarely be considered a violation of any decent provider’s policies.

Not enough data to be actionable

Generally speaking, email service providers want to see an email’s body and unredacted headers. That makes for a well-formed and fully actionable complaint. With that information, we can:

  1. Establish that the message came from a system over which we have oversight
  2. Determine which customer sent the message (in other words, “Who, exactly, are we investigating?”)
  3. Determine that the message was sent within an actionable time frame.
Read morePolicy at scale: The purpose of a policy is protection

Sometimes, server log files are also extremely appropriate as evidence. This is especially true in the case where a customer may be sending messages to many non-existent addresses. Since those messages will all bounce, there are literally no headers to tender, but the data in the log file can give me the same bits of information and provide a good springboard into an investigation of a customer’s practices.

In this context, it does help to think of policy enforcement like you do the police. You can’t go to the police and tell them about a bank robbery that you think was committed in a completely different country half the world away. That’s not within their jurisdiction. While they might be able to assist with getting you in contact with the right person to handle a complaint about an issue the next town over, the further away from their jurisdiction that you get, the less likely that you should assume that they will be able to assist you.

The least actionable complaint

The least actionable complaint is the one where the person writes in and says, “You have a spammer on your network,” “Customer_name is a spammer,” or “Customer_name is violating some_law” without any proof. Policy enforcement does want to know all of those things, but our currency is evidence, and this is a request for me to task resources to the generation of evidence-based merely upon the assertion that a particular customer has (or is) a problem. Unfortunately, I don’t know if the problem is with one list or if the customer’s entire business model has them operating in violation of my policy. How do I determine which one if it’s only a single list? When I start from a single, known list, I can work my way backward from smaller questions to larger ones.

So, when the issue comes down to “devote resources to go on a fishing expedition” or “devote resources to investigate complaints with data,” it’s a really easy call to make.

The “GWF” and threshold issues

I almost labeled this one as the easiest to close. But, until you get used to them, they take a level of investigation and can confuse you. The acronym “GWF” stands for “Goober With Firewall.” It refers to complaints from people who generally don’t know how servers work with log files attached to the complaint showing normal network transactions occurring. Usually, these will come in with breathless notes saying that a mail server has been attacking a name server on port 53 by looking up MX and A records. (Hint: That’s how a sending mail server discovers where to find the receiving mail server.) Or, the breathless note will talk about a mail server attacking another mail server by connecting to it on port 25 and sending mail. (Hint: that’s what mail servers exist to do.)

If the issue is that a server is occupying all available resources, then that should be stated. If the issue is that a server is sending mail too quickly, opening too many simultaneous connections, or some other issue, then that should be stated. But if all you do is tell me that mail servers are doing things that they are supposed to do, then I’m going to close that case immediately because it does not show me anything actionable.

A word about time frames

This one is more of a matter of taste than anything else. Abuse desks deal with a large number of cases, and that’s important to remember. Customers who were doing bad things a year ago have either learned their lesson and changed, or they’re still doing bad things today and are continuing to get new complaints. That places a freshness bound on incoming complaints.

Depending on the needs of the abuse desk and how busy they are overall, this number can be quite elastic. I think that all other things being equal, I want to start an investigation into a mailing within two weeks of receiving it from the person complaining. In my experience, when you push out past about a week, customers ask, “why did it take so long to contact us about this?” You can generally get another week by pointing out case volumes and backlog. But, the complaints become more vociferous when you get past that two-week point and spend more time defending the lag than you are investigating the case. And, as the saying goes, “spammers gonna spam.” So, there will always be more spam if there is a genuine issue.

Nevertheless, I feel compelled to note that I know other desks operating on different scales. One has the headcount to set their marker at around 4 days, while another pushes things out to a month.

Conclusion

When working at scale, an abuse desk depends upon complainants to provide enough information to prove — not merely assert — that the actions complained about

  • Recently happened
  • From a machine that the abuse desk has jurisdiction over
  • As the result of an action that an identifiable customer took.

That information usually takes the form of a message with routing headers. But, it’s also possible to provide a log file if a sufficient explanation is included to say how or why the network traffic depicted is not normal traffic.

For More Information

Learn how to find headers

Mickey Chandler
Latest posts by Mickey Chandler (see all)