Table Of Contents

Previous topic

Normalization

This Page

WebAPI

Versioning

To maintain backwards compatibility the REST path will be prefixed with version numbers, a prefix of /api/d/ will serve the current unstable development api, whereas /api/v* will serve the specified stable version.

Examples:

/api/v1/sessions
Sessions resource as it was defined in release 1.
/api/v2/sessions
Sessions resource as it was defined in release 2.
/api/d/sessions
Sessions resource under development.

Path conventions

To ensure a clean API, Mnemosyne has the following conventions for URL construction:

  1. If the consumer has filtering options use query strings.
  2. If the consumer has no options in regards to filtering use a clean path.

Access levels

Level Description
100 Administrator
70 Access to all data, including raw hpfeeds store
60 Access to normalized data where IP of honeypots has been removed.
10 Public access to basic sanitized data. (Example: Dorks service for glastopf)

Resources as of version 1

HPFeeds

The HPFeeds resource located at /api/v1/hpfeeds serves unparsed data from various HPFeed channels.

GET /api/v1/hpfeeds

Returns filtered HPFeeds data.

Example request:

GET /api/v1/hpfeeds?channel=glastopf.events HTTP/1.1
Accept: application/json, text/javascript

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "hpfeeds": [
    {
      "ident": "2l3if@hp8",
      "timestamp": "2012-12-26T20:01:46.908000",
      "normalized": true,
      "_id": "5deadbeef0e0f7b874e8088d",
      "payload": "<payload>",
      "channel": "glastopf.events"
    },
    {
      "ident": "2l3if@hp8",
      "timestamp": "2012-12-26T20:05:32.773000",
      "normalized": false,
      "_id": "50db57aadfe0f7b8deadbeef",
      "payload": "<payload>",
      "channel": "glastopf.events"
    }
  ]
}
Query Parameters:
 
  • id – hpfeed’s unique id (optional).
  • channel – channel name (optional).
  • limit – limit number of returned items (optional, default is 50).
Status Codes:
  • 200 – no error.
  • 400 – Bad request.

HPFeeds statistics

The HPFeeds statistics resources is a subresource and is located at /api/v1/hpfeeds/stats. .. http:get:: /api/v1/hpfeeds/stats

Returns a collection of hourly counts for one or more HPFeeds channels.

Example request:

GET /api/v1/hpfeeds/stats?channel=dionaea.capture HTTP/1.1
Accept: application/json, text/javascript

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{'stats':
 [
  {'hourly': {'12': 1, '13': 2}, 'date': '20130906', 'channel': 'dionaea.capture'},
  {'hourly': {'13': 115, '12': 1978}, 'date': '20130907', 'channel': 'dionaea.capture'}
 ]
}
query channel:channel name.
query date:limit query to an specific date, format: YYYYMMDD, example: 20131230.
statuscode 200:no error.
statuscode 400:Bad request.
GET /api/v1/hpfeeds/stats/total

Returns the total count for every received channel name.

Example request:

GET /api/v1/hpfeeds/stats/total HTTP/1.1
Accept: application/json, text/javascript

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{'stats':
 [
  {'channel': 'dionaea_capture', 'count': 22},
  {'channel': 'mwbinary_dionaea_sensorunique', 'count': 1}
 ]
}
Status Codes:
  • 200 – no error.
  • 400 – Bad request.

Sessions

The Sessions resource located at /api/v1/sessions contains normalized data from traditional serverside honeypots.

GET /api/v1/sessions

Returns sessions filtered by query parameters.

Example request:

GET /api/v1/sessions?honeypot=kippo&source_port=36888 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "sessions": [
    {
      "protocol": "ssh",
      "hpfeed_id": "50dc4244dfe0f7bf93d06076",
      "timestamp": "2012-12-27T12:42:44.296000",
      "source_ip": "181.5.23.53",
      "session_ssh": {
        "version": "SSH-2.0-libssh-0.1"
      },
      "source_port": 36868,
      "destination_port": 2222,
      "_id": "50dcc2ebdfe0f7c4d1ce350d",
      "honeypot": "Kippo",
      "auth_attempts": [
        {
          "login": "root",
          "password": "321muie321"
        }
      ]
    },
    {
      "protocol": "ssh",
      "hpfeed_id": "50dc4249dfe0f7bf93d06077",
      "timestamp": "2012-12-27T12:42:49.131000",
      "source_ip": "182.5.23.53",
      "session_ssh": {
        "version": "SSH-2.0-libssh-0.1"
      },
      "source_port": 36868,
      "destination_port": 2222,
      "_id": "50dcc2ebdfe0f7c4d1ce350e",
      "honeypot": "Kippo",
      "auth_attempts": [
        {
          "login": "root",
          "password": "123muie123"
        }
      ]
    }
  ]
}
Query Parameters:
 
  • id – unique identifer (optional).
  • protocol – protocol name - ssh, imap, etc (optional).
  • honeypot – honeypot type - kippo, dionaea, glastopf, etc (optional).
  • source_ip – ip address of attacker (optional).
  • source_port – tcp port of attacker (optional).
  • destination_ip – ip address of honeypot (optional).
  • destination_port – tcp port of honeypot (optional).
  • limit – limit number of returned items (optional, default is 50).
Status Codes:
  • 200 – no error.
  • 400 – Bad request.

URLS

The URLS resource located at /urls, contains urls which potentially are serving malicious content.

GET /api/v1/urls

Returns urls serving potential malicious content. If any files has been extracted, an reference to the checksum will be provided.

Example request:

GET /api/v1/urls?url_regex=\.ru(\/|\:|$) HTTP/1.1
Host: example.com
Accept: application/json

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{
 "urls": [
   {
     "url": "http://ikbyznod.ru/count26.php",
     "_id": "50ec7f6fc1512da84f113386",
     "extractions": [
       {
         "timestamp": "2012-12-26T13:51:13.507000",
         "hashes": {
           "md5": "549eccb6939274ac9664f0201e4771c4",
           "sha1": "d337b47020b1e214d35b044483bf04ae1f0a7b4d",
           "sha512": "53ece48162e635bd93ea3240c12b4a844974de0a75f3b30da1f18f8e2892c10a9930a2380673afd4521083b9f952a10b3c54de3be477ab1f11c61a8902c0d435"
         }
       }
     ],
     "hpfeeds_ids": [
       "50da8260dfe0f7b2c68c2fde"
     ]
   },
   {
     "url": "http://www.ajy-aa.xx/images/M_images/t?%0D?",
     "_id": "50ec7f70c1512da84f113387",
     "hpfeeds_ids": [
       "50dad02bdfe0f7b4f48cd434",
       "50dad0a6dfe0f7b4f48cd435"
     ]
   },
   {
     "url": "http://www.xxyycatab.com.qq",
     "_id": "50ec7f70c1512da84f113388",
     "hpfeeds_ids": [
       "50dada38dfe0f7b53ceb8383"
     ]
   }
 ]
}
Query Parameters:
 
  • url_regex – PCRE regex which will be tried against the stored url (optional).
  • hash – returns URL’s where the files with the specified HASH has been downloaded (optional).
  • limit – limit number of returned items (optional, default is 50).
Status Codes:
  • 200 – no error.
  • 400 – Bad request.

Files

The Files resource located at /api/v1/files contains various forms of binaries and code samples collected from HPFeeds channels.

GET /api/v1/files

Returns matches for the given hash. The following hashes are supported: MD5, SHA1, SHA512

Example request:

GET /api/v1/files?hash=549eccb6939274ac9664f0201e4771c4 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

  {
    "files": [
      {
        "_id": "50e5e440cfd26d1f23bfe7b7",
        "content_guess": "Javascript",
        "data": "0a0909090909706172656e742e6c6f636174696f6e2e68726566203d2022687474703a2f2f736f6e617464616e69736d616e6c696b2e636f6d2f6d61696e6c792e68746d6c223b0a09090909",
        "encoding": "hex",
        "hashes": {
          "md5": "549eccb6939274ac9664f0201e4771c4",
          "sha1": "d337b47020b1e214d35b044483bf04ae1f0a7b4d",
          "sha512": "53ece48162e635bd93ea3240c12b4a844974de0a75f3b30da1f18f8e2892c10a9930a2380673afd4521083b9f952a10b3c54de3be477ab1f11c61a8902c0d435"
        },
        "hpfeed_ids": [
          "50da8260dfe0f7b2c68c2fde"
        ]
      }
    ]
  }
Query Parameters:
 
  • hash – SHA1, SHA51 or MD5 digest (required).
  • no_data – If present the ‘data’ field will not be returned (optional).
Status Codes:
  • 200 – no error.
  • 400 – Bad request.
GET /api/v1/aux/dorks

Serves Dorks collected by Glastopf.

Example request:

GET /api/v1/aux/dorks HTTP/1.1
Accept: application/json, text/javascript

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{
 "dorks": [
  {
   "content": "/pivotx/includes/index.php",
   "count": 716,
   "firsttime": "2013-02-01T20:38:42+00:00",
   "lasttime": "2013-01-14T16:20:51.504000",
   "type": "inurl"
  },
  {
   "content": "/axis-cgi/mjpg/wp-content/themes/diner/timthumb.php",
   "count": 545,
   "firsttime": "2013-02-01T20:38:32+00:00",
   "lasttime": "2013-01-14T16:26:03.036000",
   "type": "inurl"
  },
  {
   "content": "/board/board/include/pivotx/includes/wp-content/pivotx/includes/timthumb.php",
   "count": 493,
   "firsttime": "2013-02-01T20:39:03+00:00",
   "lasttime": "2013-01-14T10:55:50.197000",
   "type": "inurl"
  },

  <--- SNIP --- >

   ]
 }