Tagged: Curator Toggle Comment Threads | Keyboard Shortcuts

  • Tuan Pembual 19:12 on 18 September 2017 Permalink | Reply
    Tags: Curator, , ,   

    Menghapus Elasticsearch Indices dengan Curator Bag (2) 

    Dulu pernah menggunakan curator_cli untuk menghapus indices. Namun config yang lama tidak bisa digunakan untuk ES 5.2.x. Mari kita coba cara baru ini (ndak baru-baru amat sih).

    1. Install curator (via pip)
      $ pip install elasticsearch-curator
      
    2. Perintah dasar curator adalah:
      $ curator [--config CONFIG.YML] [--dry-run] ACTION_FILE.YML
      
    3. Atur configurasi dalam file config.yml
      $ cd ~
      $ vim config.yml
      

      Isinya kira-kira begini:

      ---
      # Remember, leave a key empty if there is no value.  None will be a string,
      # not a Python "NoneType"
      client:
        hosts:
          - 192.168.33.201 #ip publish es
        port: 9200
        url_prefix:
        use_ssl: False
        certificate:
        client_cert:
        client_key:
        ssl_no_validate: False
        http_auth:
        timeout: 30
        master_only: False
      
      logging:
        loglevel: INFO
        logfile:
        logformat: default
        blacklist: ['elasticsearch', 'urllib3']
      
    4. Atur aksi yang akan dilakukan dalam file delete_indices.yml
      $ vim delete_indices.yml
      

      Isinya kira-kira begini:

      ---
      # Remember, leave a key empty if there is no value.  None will be a string,
      # not a Python "NoneType"
      #
      # Also remember that all examples have 'disable_action' set to True.  If you
      # want to use this action as a template, be sure to set this to False after
      # copying it.
      actions:
        1:
          action: delete_indices
          description: >-
            Delete indices older than 7 days (based on index name), #for filebeat-
            prefixed indices. Ignore the error if the filter does not result in an
            actionable list of indices (ignore_empty_list) and exit cleanly.
          options:
            ignore_empty_list: True
            disable_action: False
          filters:
          #- filtertype: pattern
          #  kind: prefix
          #  value: filebeat-
          - filtertype: age
            source: name
            direction: older
            timestring: '%Y.%m.%d'
            unit: days
            unit_count: 7
      
    5. Testing konfigurasi curator.
      $ curator --config config.yml delete_indices.yml
      
    6. Atur penjadwalan curator dalam crontab
      $ crontab -e
      

      Isi cron kira-kira begini:

      0 6 * * * /usr/local/bin/curator --config /home/user/config.yml /home/user/delete_indices.yml
      

    Selesai.

    Bacaan:
    [0] https://www.elastic.co/guide/en/elasticsearch/client/curator/current/command-line.html

    Estu~

    Advertisements
     
  • Tuan Pembual 14:53 on 12 July 2016 Permalink | Reply
    Tags: Curator, , ,   

    Menghapus Elasticsearch Indices dengan Curator 

    10647200_10206160724766161_5644663822058867814_n

    Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots by:

    1. Obtaining the full list of indices (or snapshots) from the cluster, as the actionable list
    2. Iterate through a list of user-defined filters to progressively remove indices (or snapshots) from this actionable list as needed.
    3. Perform various actions on the items which remain in the actionable list.

    Sore kemaren kerjaan ditutup dengan peringatan serius. Space hardisk untuk menjalankan ELK Stack hampir habis dikarenakan penuh oleh log dari filebeat. Kudu dibersihkan. Tool yang paling mudah adalah menggunakan Curator. Tool dari ES yang ditulis dengan python.

    Eksekusi:

    1. Login ke mesin remote
    2. Install Curator lewat pip
      $ pip install elasticsearch-curator
      
    3. Jalankan perintah untuk hapus indices sampai 30 hari terakhir
      $ curator -h
      $ curtor --host 127.0.0.1 delete indices --older-than 30 --time-unit days --timestring '%Y.%m.%d'
      
    4. Atur agar berjalan teratur melalu cron agak eksekusi setiap Senin jam 10 pagi.
      $ sudo crontab -e

      Isi dengan:

      0 10 * * 2 /usr/local/bin/curator --host 127.0.0.1 delete indices --older-than 30 --time-unit days --timestring '%Y.%m.%d'
      
    5. Lakukan restart cron biar yakin.
      $ sudo service cron restart
      

    *Ps: sebelumnya saya agak bingung. Dari webnya ES perintahnya beda. Sebelumnya saya memasang Curator di laptop saya. Trus lakukan konfigurasi (./curator/curator.yml) dan membuat action file (./curator/delete_indices.yml) tapi malah ndak jalan. Eh ternyata pake cara lain.

    Semoga ada manfaatnya.

    Estu

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel