merge: with pam (#14911)

* perf: change i18n

* perf: pam

* perf: change translate

* perf: add check account

* perf: add date field

* perf: add account filter

* perf: remove some js

* perf: add account status action

* perf: update pam

* perf: 修改 discover account

* perf: update filter

* perf: update gathered account

* perf: 修改账号同步

* perf: squash migrations

* perf: update pam

* perf: change i18n

* perf: update account risk

* perf: 更新风险发现

* perf: remove css

* perf: Admin connection token

* perf: Add a switch to check connectivity after changing the password, and add a custom ssh command for push tasks

* perf: Modify account migration files

* perf: update pam

* perf: remove to check account dir

* perf: Admin connection token

* perf: update check account

* perf: 优化发送结果

* perf: update pam

* perf: update bulk update create

* perf: prepaire using thread timer for bulk_create_decorator

* perf: update bulk create decorator

* perf: 优化 playbook manager

* perf: 优化收集账号的报表

* perf: Update poetry

* perf: Update Dockerfile with new base image tag

* fix: Account migrate 0012 file

* perf: 修改备份

* perf: update pam

* fix: Expand resource_type filter to include raw type

* feat: PAM Service (#14552)

* feat: PAM Service

* perf: import package name

---------

Co-authored-by: jiangweidong <1053570670@qq.com>

* perf: Change secret dashboard (#14551)

Co-authored-by: feng <1304903146@qq.com>

* perf: update migrations

* perf: 修改支持 pam

* perf: Change secret record table dashboard

* perf: update status

* fix: Automation send report

* perf: Change secret report

* feat: windows accounts gather

* perf: update change status

* perf: Account backup

* perf: Account backup report

* perf: Account migrate

* perf: update service to application

* perf: update migrations

* perf: update logo

* feat: oracle accounts gather (#14571)

* feat: oracle accounts gather

* feat: sqlserver accounts gather

* feat: postgresql accounts gather

* feat: mysql accounts gather

---------

Co-authored-by: wangruidong <940853815@qq.com>

* feat: mongodb accounts gather

* perf: Change secret

* perf: Migrate

* perf: Merge conflicting migration files

* perf: Change secret

* perf: Automation filter org

* perf: Account push

* perf: Random secret string

* perf: Enhance SQL query and update risk handling in accounts

* perf: Ticket filter assignee_id

* perf: 修改 account remote

* perf: 修改一些 adhoc 任务

* perf: Change secret

* perf: Remove push account extra api

* perf: update status

* perf: The entire organization can view activity log

* fix: risk field check

* perf: add account details api

* perf: add demo mode

* perf: Delete gather_account

* perf: Perfect solution to account version problem

* perf: Update status action to handle multiple accounts

* perf: Add GatherAccountDetailField and update serializers

* perf: Display account history in combination with password change records

* perf: Lina translate

* fix: Update mysql_filter to handle nested user info

* perf: Admin connection token validate_permission account

* perf: copy move account

* perf: account filter risk

* perf: account risk filter

* perf: Copy move account failed message

* fix: gather account sync account to asset

* perf: Pam dashboard

* perf: Account dashboard total accounts

* perf: Pam dashboard

* perf: Change secret filter account secret_reset

* perf: 修改 risk filter

* perf: pam translate

* feat: Check for leaked duplicate passwords. (#14711)

* feat: Check for leaked duplicate passwords.

* perf: Use SQLite instead of txt as leak password database

---------

Co-authored-by: jiangweidong <1053570670@qq.com>
Co-authored-by: 老广 <ibuler@qq.com>

* perf: merge with remote

* perf: Add risk change_password_add handle

* perf: Pam dashboard

* perf: check account manager import

* perf: 重构扫描

* perf: 修改 db

* perf: Gather account manager

* perf: update change db lib

* perf: dashboard

* perf: Account gather

* perf: 修改 asset get queryset

* perf: automation report

* perf: Pam account

* perf: Pam dashboard api

* perf: risk add account

* perf: 修改 risk check

* perf: Risk account

* perf: update risk add reopen action

* perf: add pylintrc

* Revert "perf: automation report"

This reverts commit 22aee54207.

* perf: check account engine

* perf: Perf: Optimism Gather Report Style

* Perf: Remove unuser actions

* Perf: Perf push account

* perf: perf gather account

* perf: Automation report

* perf: Push account recorder

* perf: Push account record

* perf: Pam dashboard

* perf: perf

* perf: update intergration

* perf: integrations application detail add account tab page

* feat: Custom change password supports configuration of interactive items

* perf: Go and Python demo code

* perf: Custom secret change

* perf: add user filter

* perf: translate

* perf: Add demo code docs

* perf: update some i18n

* perf: update some i18n

* perf: Add Java, Node, Go, and cURL demo code

* perf: Translate

* perf: Change secret translate

* perf: Translate

* perf: update some i18n

* perf: translate

* perf: Ansible playbook

* perf: update some choice

* perf: update some choice

* perf: update account serializer remote unused code

* perf: conflict

* perf: update import

---------

Co-authored-by: ibuler <ibuler@qq.com>
Co-authored-by: feng <1304903146@qq.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: wangruidong <940853815@qq.com>
Co-authored-by: jiangweidong <1053570670@qq.com>
Co-authored-by: feng626 <57284900+feng626@users.noreply.github.com>
Co-authored-by: zhaojisen <1301338853@qq.com>

apps/accounts/filters.py

@@ -1,36 +1,116 @@
 # -*- coding: utf-8 -*-
 #
 from django.db.models import Q
+from django.utils import timezone
 from django_filters import rest_framework as drf_filters
 
 from assets.models import Node
 from common.drf.filters import BaseFilterSet
-from .models import Account, GatheredAccount, ChangeSecretRecord
+from common.utils.timezone import local_zero_hour, local_now
+from .models import Account, GatheredAccount, ChangeSecretRecord, PushSecretRecord, IntegrationApplication
 
 
 class AccountFilterSet(BaseFilterSet):
-    ip = drf_filters.CharFilter(field_name='address', lookup_expr='exact')
-    hostname = drf_filters.CharFilter(field_name='name', lookup_expr='exact')
-    username = drf_filters.CharFilter(field_name="username", lookup_expr='exact')
-    address = drf_filters.CharFilter(field_name="asset__address", lookup_expr='exact')
-    asset_id = drf_filters.CharFilter(field_name="asset", lookup_expr='exact')
-    asset = drf_filters.CharFilter(field_name='asset', lookup_expr='exact')
-    assets = drf_filters.CharFilter(field_name='asset_id', lookup_expr='exact')
-    nodes = drf_filters.CharFilter(method='filter_nodes')
-    node_id = drf_filters.CharFilter(method='filter_nodes')
-    has_secret = drf_filters.BooleanFilter(method='filter_has_secret')
-    platform = drf_filters.CharFilter(field_name='asset__platform_id', lookup_expr='exact')
-    category = drf_filters.CharFilter(field_name='asset__platform__category', lookup_expr='exact')
-    type = drf_filters.CharFilter(field_name='asset__platform__type', lookup_expr='exact')
+    ip = drf_filters.CharFilter(field_name="address", lookup_expr="exact")
+    hostname = drf_filters.CharFilter(field_name="name", lookup_expr="exact")
+    username = drf_filters.CharFilter(field_name="username", lookup_expr="exact")
+    address = drf_filters.CharFilter(field_name="asset__address", lookup_expr="exact")
+    asset_id = drf_filters.CharFilter(field_name="asset", lookup_expr="exact")
+    asset = drf_filters.CharFilter(field_name="asset", lookup_expr="exact")
+    assets = drf_filters.CharFilter(field_name="asset_id", lookup_expr="exact")
+    nodes = drf_filters.CharFilter(method="filter_nodes")
+    node_id = drf_filters.CharFilter(method="filter_nodes")
+    has_secret = drf_filters.BooleanFilter(method="filter_has_secret")
+    platform = drf_filters.CharFilter(
+        field_name="asset__platform_id", lookup_expr="exact"
+    )
+    category = drf_filters.CharFilter(
+        field_name="asset__platform__category", lookup_expr="exact"
+    )
+    type = drf_filters.CharFilter(
+        field_name="asset__platform__type", lookup_expr="exact"
+    )
+    latest_discovery = drf_filters.BooleanFilter(method="filter_latest")
+    latest_accessed = drf_filters.BooleanFilter(method="filter_latest")
+    latest_updated = drf_filters.BooleanFilter(method="filter_latest")
+    latest_secret_changed = drf_filters.BooleanFilter(method="filter_latest")
+    latest_secret_change_failed = drf_filters.BooleanFilter(method="filter_latest")
+    risk = drf_filters.CharFilter(
+        method="filter_risk",
+    )
+    integrationapplication = drf_filters.CharFilter(method="filter_integrationapplication")
+    long_time_no_change_secret = drf_filters.BooleanFilter(method="filter_long_time")
+    long_time_no_verified = drf_filters.BooleanFilter(method="filter_long_time")
 
     @staticmethod
     def filter_has_secret(queryset, name, has_secret):
-        q = Q(secret__isnull=True) | Q(secret='')
+        q = Q(_secret__isnull=True) | Q(_secret="")
         if has_secret:
             return queryset.exclude(q)
         else:
             return queryset.filter(q)
 
+    @staticmethod
+    def filter_long_time(queryset, name, value):
+        date = timezone.now() - timezone.timedelta(days=30)
+
+        if name == "long_time_no_change_secret":
+            field = "date_change_secret"
+            confirm_field = "change_secret_status"
+        else:
+            field = "date_verified"
+            confirm_field = "connectivity"
+
+        q = Q(**{f"{field}__lt": date}) | Q(**{f"{field}__isnull": True})
+        confirm_q = {f"{confirm_field}": "na"}
+        queryset = queryset.exclude(**confirm_q).filter(q)
+        return queryset
+
+    @staticmethod
+    def filter_risk(queryset, name, value):
+        if not value:
+            return queryset
+
+        queryset = queryset.filter(risks__risk=value)
+        return queryset
+
+    @staticmethod
+    def filter_integrationapplication(queryset, name, value):
+        if not value:
+            return queryset
+
+        integrationapplication = IntegrationApplication.objects.filter(pk=value).first()
+        if not integrationapplication:
+            return IntegrationApplication.objects.none()
+        queryset = integrationapplication.get_accounts()
+        return queryset
+
+    @staticmethod
+    def filter_latest(queryset, name, value):
+        if not value:
+            return queryset
+
+        date = timezone.now() - timezone.timedelta(days=7)
+        kwargs = {}
+
+        if name == "latest_discovery":
+            kwargs.update({"date_created__gte": date, "source": "collected"})
+        elif name == "latest_accessed":
+            kwargs.update({"date_last_login__gte": date})
+        elif name == "latest_updated":
+            kwargs.update({"date_updated__gte": date})
+        elif name == "latest_secret_changed":
+            kwargs.update({"date_change_secret__gt": date})
+
+        if name == "latest_secret_change_failed":
+            queryset = queryset.filter(date_change_secret__gt=date).exclude(
+                change_secret_status="ok"
+            )
+
+        if kwargs:
+            queryset = queryset.filter(date_last_login__gte=date)
+        return queryset
+
     @staticmethod
     def filter_nodes(queryset, name, value):
         nodes = Node.objects.filter(id=value)
@@ -40,19 +120,22 @@ class AccountFilterSet(BaseFilterSet):
         node_qs = Node.objects.none()
         for node in nodes:
             node_qs |= node.get_all_children(with_self=True)
-        node_ids = list(node_qs.values_list('id', flat=True))
+        node_ids = list(node_qs.values_list("id", flat=True))
         queryset = queryset.filter(asset__nodes__in=node_ids)
         return queryset
 
     class Meta:
         model = Account
-        fields = ['id', 'asset', 'source_id', 'secret_type']
+        fields = ["id", "asset", "source_id", "secret_type", "category", "type"]
 
 
 class GatheredAccountFilterSet(BaseFilterSet):
-    node_id = drf_filters.CharFilter(method='filter_nodes')
-    asset_id = drf_filters.CharFilter(field_name='asset_id', lookup_expr='exact')
-    asset_name = drf_filters.CharFilter(field_name='asset__name', lookup_expr='icontains')
+    node_id = drf_filters.CharFilter(method="filter_nodes")
+    asset_id = drf_filters.CharFilter(field_name="asset_id", lookup_expr="exact")
+    asset_name = drf_filters.CharFilter(
+        field_name="asset__name", lookup_expr="icontains"
+    )
+    status = drf_filters.CharFilter(field_name="status", lookup_expr="exact")
 
     @staticmethod
     def filter_nodes(queryset, name, value):
@@ -60,14 +143,38 @@ class GatheredAccountFilterSet(BaseFilterSet):
 
     class Meta:
         model = GatheredAccount
-        fields = ['id', 'username']
+        fields = ["id", "username"]
 
 
-class ChangeSecretRecordFilterSet(BaseFilterSet):
-    asset_name = drf_filters.CharFilter(field_name='asset__name', lookup_expr='icontains')
-    account_username = drf_filters.CharFilter(field_name='account__username', lookup_expr='icontains')
-    execution_id = drf_filters.CharFilter(field_name='execution_id', lookup_expr='exact')
+class SecretRecordMixin:
+    asset_name = drf_filters.CharFilter(
+        field_name="asset__name", lookup_expr="icontains"
+    )
+    account_username = drf_filters.CharFilter(
+        field_name="account__username", lookup_expr="icontains"
+    )
+    execution_id = drf_filters.CharFilter(
+        field_name="execution_id", lookup_expr="exact"
+    )
+    days = drf_filters.NumberFilter(method="filter_days")
 
+    @staticmethod
+    def filter_days(queryset, name, value):
+        value = int(value)
+
+        dt = local_zero_hour()
+        if value != 1:
+            dt = local_now() - timezone.timedelta(days=value)
+        return queryset.filter(date_finished__gte=dt)
+
+
+class ChangeSecretRecordFilterSet(SecretRecordMixin, BaseFilterSet):
     class Meta:
         model = ChangeSecretRecord
-        fields = ['id', 'status', 'asset_id', 'execution']
+        fields = ["id", "status", "asset_id", "execution"]
+
+
+class PushAccountRecordFilterSet(SecretRecordMixin, BaseFilterSet):
+    class Meta:
+        model = PushSecretRecord
+        fields = ["id", "status", "asset_id", "execution"]