When scapy gets bored

Do you remember “Star Wars Traceroute”? Using VRFs and DNS server the Star Wars story is told.

I wrote a small script in python as a PoC that it can be done with scapy without any routing on a single machine. Just for fun.

Traceroute shows the path to the host based on Time Exceed ICMP packets from routers on the path. I have decided that scapy can send these fake replies from some additional IP on a single host.

I used IPv6, 'cause Digital Ocean does not allow to add additional IPv4. It will perfectly work for IPv4 as well.


Usage:

1. block icmp6 from the desired host (or all host) via ip6tables
2. run the script

Here is how mtr looks like to spron.in when script is enabled:

  3.|-- 100ge6-2.core1.ams1.he.ne  0.0%     2   12.5  23.2  12.5  33.9  15.1
  4.|-- 2001:7f8:1:0:a500:20:130:  0.0%     2    9.0   9.1   9.0   9.2   0.0
  5.|-- ???                       100.0     2    0.0   0.0   0.0   0.0   0.0
  6.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   18.4  27.4  18.4  36.3  12.6
  7.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   19.9  18.3  16.6  19.9   2.2
  8.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   28.2  24.5  20.7  28.2   5.3
  9.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   17.9  18.8  17.9  19.7   1.0
 10.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   21.1  22.7  21.1  24.3   2.2
 11.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   20.8  22.7  20.8  24.6   2.6
 12.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   23.5  22.2  20.9  23.5   1.7
 13.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   21.4  21.6  21.4  21.8   0.0
 14.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   20.7  20.8  20.7  20.8   0.0
 15.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   26.1  24.5  23.0  26.1   2.0
 16.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   23.4  23.1  22.9  23.4   0.0
 17.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   20.4  22.3  20.4  24.2   2.6
 18.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   19.4  21.3  19.4  23.3   2.6
 19.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   21.3  21.9  21.3  22.5   0.0
 20.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   20.9  22.2  20.9  23.4   1.7
 21.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   20.1  20.9  20.1  21.6   1.0
 22.|-- 2a03:b0c0:0:1010::32:2003  0.0%     2   16.7  18.9  16.7  21.2   3.0
 23.|-- 2a03:b0c0:0:1010::32:2002  0.0%     2   35.9  25.9  15.9  35.9  14.1

2a03:b0c0:0:1010::32:2003 is an additional IP on my VPS, 2a03:b0c0:0:1010::32:2002 - real one.

Have a look at the script on github.

Multicast snooping and bridges

If you are running two containers (LXC, Docker, whatever) on the same machine, and you use IPv6 there, then you might notice, that these machines do not see each other.

You do know about neighbor discovery in IPv6, right? No ARP and all the bells and whistles.

Remember, no IGMP here, just MLD (Multicast Listener Discovery). These two a pretty the same, but there are some issues with MLD in Linux.

Some say that you should disable multicast_snooping on the bridge. It can be done via sysfs:

echo -n 0 > /sys/class/net/%BRIDGE_NAME%/bridge/multicast_snooping

Well, that will work. But you should understand, that now all your network behind that bridge will be flooded by multicast packets. Okay for small installations.

Instead, enable multicast querier:

echo -n 1 /sys/class/net/%BRIDGE_NAME%/bridge/multicast_querier

Querier will update snooping filters by sending broadcast queries.

After that you will need to remove and add again an IPv6 address in your container, so MLD query would be resent.

I would strongly suggest you not to disable multicast snooping. That might be a security flaw and flood your network in a second.

German attack

Have a look at awesome write up about vulnerable DSL routers in Germany: https://isc.sans.edu/forums/diary/Port+7547+SOAP+Remote+Code+Execution+Attack+Against+DSL+Modems/21759/

Looks like another round for Mirai.

Be aware, port 7547 has nothing to do with TR-069, but some kind of a lame webserver.

P.S. If the bad guys were a bit more discrete and not scanning all over the place, that attack might have had a huge impact.

IPv6 in linux cluster

Imagine quite a usual scheme - bunch of servers in a cluster for web.
It may be two servers with failover IP (keepalived, heartbeat) or a cluster (corosync).

In a scheme like that, to avoid scripting, you probably set the following sysctl value:

net.ipv4.ip_nonlocal_bind=1

Awesomely simple and handy, your favorite web server is able to bind to virtual address, even if it is not on the server now. Cool, right?  

Rotten apple

Beware debian users and all others whose kernel version is <4.3. And I know, there are many of you  out there. Beware, 'cause this trick won't work on IPv6. The same sysctl option for IPv6 (net.ipv6.ip_nonlocal_bind) was introduced just in kernel 4.3.

Here is the commit to Documentation/networking/ip-sysctl.txt: https://github.com/torvalds/linux/commit/35a256fee52c7c207796302681fa95189c85b408 (July 8, 2015)

Interesting thing, almost the same commit was rejected in 2011: http://patchwork.ozlabs.org/patch/130748/

What to do

1. Upgrade. Version 4.3 was introduced more then a year ago. Debian pain.
2. Use balancers, move all cluster-fancy-party to one level up in your infrastructure. Keepalived + IPVS is a good choice. It does not bind. Well it does, but you don't need to bind to an absent IP.
3. Scripting (brrr).

python "in" operator

Hello comrade,

recently I had smth like this in my code:

>>> string = 'hdfjghweurj'
>>> sub = 'ghw'
>>> sub in string
True

I have asked myself - how "in" is working? This knowledge might be useless, but knowing how to get this info might help you understand how language works and what algorithms does it use. All examples below are for python 2, basically in python 3 it is pretty the same, but some methods differ.

Here are the steps:

dis

First, lets disassemble.

>>> import dis
>>> def foo():
...     'aaa' in []
... 
>>> dis.dis(foo)
  2           0 LOAD_CONST               1 ('aaa')
              3 LOAD_CONST               2 (())
              6 COMPARE_OP               6 (in)
              9 POP_TOP             
             10 LOAD_CONST               0 (None)
             13 RETURN_VALUE        

We now know, that it is a COMPARE_OP. Let's dive into sources.

ceval.c

Let's have a look at ceval.c loop and find our COMPARE_OP:

        
        TARGET(COMPARE_OP) {
            ...
            else {
              slow_compare:
                x = cmp_outcome(oparg, v, w);
            }
            ...
        }

Looks like all logic is in cmp_outcome. It can be found in the same file, what we interested in is:

      case PyCmp_IN:
        res = PySequence_Contains(w, v);
        if (res < 0)
            return NULL;
        break;

PySequence_Contains is interesting for us.

Objects

Searching the repo we will land at Objects/abstract.c:

int
PySequence_Contains(PyObject *seq, PyObject *ob)
{
    Py_ssize_t result;
    if (PyType_HasFeature(seq->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) {
        PySequenceMethods *sqm = seq->ob_type->tp_as_sequence;
        if (sqm != NULL && sqm->sq_contains != NULL)
            return (*sqm->sq_contains)(seq, ob);
    }
    result = _PySequence_IterSearch(seq, ob, PY_ITERSEARCH_CONTAINS);
    return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
}

PySequenceMethods is our guy. As we are talking about strings, we should look for them. This leads us to Objects/stringobject.c:

Objects/stringobject.c:    (objobjproc)string_contains /*sq_contains*/

string_contains could be found in the same file:

...
return stringlib_contains_obj(str_obj, sub_obj);

Now,  of course, we must go to Objects/stringlib/find.h. Not looking through the source, at the header you might see, that it uses fastsearch.h:

#ifndef STRINGLIB_FASTSEARCH_H
#error must include "stringlib/fastsearch.h" before including this module
#endif

Lets have a look at it. The first comment states:

/* fast search/count implementation, based on a mix between boyer-
   moore and horspool, with a few more bells and whistles on the top.
   for some more background, see: http://effbot.org/zone/stringlib.htm */

That is how you know, that "in" operator in Python, when used for strings, uses Boyer-Moore-Horspool algorithm, with some "bells and whistles". 

spron.in paranoid edition

Hello comrade,

on the 26th of October I wrote a post on HSTS preload. Also on that date I have requested to add spron.in to preload list. And finally:

Took almost a month :)

From mirai sources

As you might know, recent huge DDoS attacks (665 Gbit/s) were organized with the help of Mirai malware. Here are some insights into its source code.

Intro

Short overview:

• Written mainly in C, but some scripts are in golang
• Loader, payload (bot) and CNC itself can be found in sources
• IPv6 not supported :( 

Russian trace
This is funny, but in admin.go file there are some Russian sentences:

admin.go: this.conn.Write([]byte("\033[34;1mпользователь\033[33;3m: \033[0m"))
admin.go: this.conn.Write([]byte("\033[34;1mпароль\033[33;3m: \033[0m"))
admin.go: this.conn.Write(append([]byte("\r\033[37;1mпроверив счета... \033[31m"), spinBuf[i % len(spinBuf)]))
admin.go: this.conn.Write([]byte("\r\033[32;1mпроизошла неизвестная ошибка\r\n"))
admin.go: this.conn.Write([]byte("\033[31mнажмите любую клавишу для выхода. (any key)\033[0m"))

How did this happen? All other comments and output info are in English.

Random scan

Mirai authors implemented its own scanning technique, that is very fast, but interesting thing, that it picks IP addresses randomly:

static ipv4_t get_random_ip(void)
{
    uint32_t tmp;
    uint8_t o1, o2, o3, o4;

    do
    {
        tmp = rand_next();

        o1 = tmp & 0xff;
        o2 = (tmp >> 8) & 0xff;
        o3 = (tmp >> 16) & 0xff;
        o4 = (tmp >> 24) & 0xff;
    }
    while (o1 == 127 ||                             // 127.0.0.0/8      - Loopback
          (o1 == 0) ||                              // 0.0.0.0/8        - Invalid address space
          (o1 == 3) ||                              // 3.0.0.0/8        - General Electric Company
          (o1 == 15 || o1 == 16) ||                 // 15.0.0.0/7       - Hewlett-Packard Company
          (o1 == 56) ||                             // 56.0.0.0/8       - US Postal Service
          (o1 == 10) ||                             // 10.0.0.0/8       - Internal network
          (o1 == 192 && o2 == 168) ||               // 192.168.0.0/16   - Internal network
          (o1 == 172 && o2 >= 16 && o2 < 32) ||     // 172.16.0.0/14    - Internal network
          (o1 == 100 && o2 >= 64 && o2 < 127) ||    // 100.64.0.0/10    - IANA NAT reserved
          (o1 == 169 && o2 > 254) ||                // 169.254.0.0/16   - IANA NAT reserved
          (o1 == 198 && o2 >= 18 && o2 < 20) ||     // 198.18.0.0/15    - IANA Special use
          (o1 >= 224) ||                            // 224.*.*.*+       - Multicast 
          (o1 == 6 || o1 == 7 || o1 == 11 || o1 == 21 || o1 == 22 || o1 == 26 || o1 == 28 || o1 == 29 || o1 == 30 || o1 == 33 || o1 == 55 || o1 == 214 || o1 == 215) // Department of Defense
    );

    return INET_ADDR(o1,o2,o3,o4);
}

Nice touch skipping GE, HP, US Post and DoD ;)

Weak and default passwords

Bot brutes weak and default passwords. There are only 63 credentials in there, all like:

add_auth_entry("\x50\x4D\x4D\x56", "\x43\x46\x4F\x4B\x4C", 8);                          // root     admin
add_auth_entry("\x43\x46\x4F\x4B\x4C", "\x43\x46\x4F\x4B\x4C", 7);                      // admin    admin
add_auth_entry("\x50\x4D\x4D\x56", "\x1A\x1A\x1A\x1A\x1A\x1A", 6);                      // root     888888

Always change default passwords.

Attack methods

    add_attack(ATK_VEC_UDP, (ATTACK_FUNC)attack_udp_generic);
    add_attack(ATK_VEC_VSE, (ATTACK_FUNC)attack_udp_vse);
    add_attack(ATK_VEC_DNS, (ATTACK_FUNC)attack_udp_dns);
    add_attack(ATK_VEC_UDP_PLAIN, (ATTACK_FUNC)attack_udp_plain);

    add_attack(ATK_VEC_SYN, (ATTACK_FUNC)attack_tcp_syn);
    add_attack(ATK_VEC_ACK, (ATTACK_FUNC)attack_tcp_ack);
    add_attack(ATK_VEC_STOMP, (ATTACK_FUNC)attack_tcp_stomp);

    add_attack(ATK_VEC_GREIP, (ATTACK_FUNC)attack_gre_ip);
    add_attack(ATK_VEC_GREETH, (ATTACK_FUNC)attack_gre_eth);

    //add_attack(ATK_VEC_PROXY, (ATTACK_FUNC)attack_app_proxy);
    add_attack(ATK_VEC_HTTP, (ATTACK_FUNC)attack_app_http);

What's interesting in attacks:

attack_udp_vse 

Attack for Valve game servers?

• Default payload: TSource Engine Query
• Default dport: 27015

attack_udp_dns

Not default amplification attack, but attack aimed to authoritative DNS servers of the targeted company. Infected IoT sends the requests to resolve random_string.target.com to ISP service provider, which forwards the request to authoritative server. IoT generates tons of these requests. 

attack_tcp_stomp

This time of attack should bypass mitigation devices. IoT device establishes the TCP connection to target and then sends ACK packets with correct sequence numbers. This keeps the connection alive. Pretty interesting, but should not normally work.

attack_gre_ip & attack_gre_eth

Two attacks with Generic Routing Encapsulation, with and without Ethernet encapsulation. GRE is a good choice, 'cause many routers bypass it without inspection.

That would be all for today. Be paranoid.

DNS infrastructure evolution

Hello comrade,

today I will show you the DNS infrastructure evolution for a project. This post is dedicated to bind 9.11 stable which was released on the 4th of October.

Few servers

Few - two or more. All registrars request at least two different name servers for a zone.

Add geoIP (with ECS)

Your project grows and you need to land customers from different countries to different frontends. Best way - use geoIP. Based on the IP-address of the client you will return different IP-addresses of your frontends.

ECS - EDNS Client Subnet - one of the extensions mechanisms (EDNS0), described in RFC 7871. Till May 2016 it was just a draft, but is widely used since 2014. If a recursive resolver with ECS feature enabled receives the request, it adds the clients IP into DNS packet and sends the request further. The authorative server returns the IP-address (or any other data) based on the client IP from DNS packet, but not the recursive resolver itself. ECS should be enabled on every server on the way of the request.

As you can see from the picture above, for the request from Chinese client via Google public NS (whose IP-address is treated by geoIP as US) the answer is still 4.3.2.1, as it should be for CN.

GeoIP and ECS are enabled easily in bind:

geoip-directory "/etc/geoip/"; // directory with maxmind .dat file
geoip-use-ecs yes;

and config views like this:

acl US { geoip country US; };
acl CN { geoip country CN; };

view US {
    match-clients { US; };
    zone "yourdomain.com." {
        type slave; 
        file "zones/us_view_yourdomain.com."; 
        masters {blabla; }; 
    };
};

view CN {
    match-clients { CN; };
    zone "yourdomain.com." {
        type slave; 
        file "zones/cn_view_yourdomain.com."; 
        masters {blabla; }; 
    };
};

Add anycast

Now you want to lower the latency between the client and DNS server. Good one. Use BGP anycast. Border routers of the ISPs will route the request to the closest node.

Do not forget to load-balance

Now take care of high-load and redundancy. Add redundant balancers and servers.

HSTS preload

Hello comrade,

everybody knows about HSTS - HTTP Strict Transport Security. It is a web security mechanism that is widely used to protect users from protocol downgrade attacks or cookie hijacking.

Works pretty simple:

1. Servers sends the header Strict-Transport-Security with the max-age parameter. 
2. Client (browser) sees the header and if it is supported - transforms all insecure links referencing the application into secured. This behavior for this site will be used for the next max-age seconds. 

See RFC 6797 for more details.

But are you paranoid enough? What happens during the very first request to your website, if it is going through HTTP? The client's browser knows nothing about HSTS settings on server's side, thus he is vulnerable.

Solution: HSTS preload. Supported by most major browsers, this technique uses the list, which is built in the browser (just like PKI certificate authorities). If your site is in this list, then the client will use only secured connection to reach it. Important to remember, that HSTS preload affects all subdomains as well (!).

To enable HSTS preload to your website, go to https://hstspreload.appspot.com . There enter your website and follow the instructions. Most likely you will have to modify your HSTS header and make it look like this:

Strict-Transport-Security: max-age=63072000; includeSubDomains; preload

Be paranoid.