1.5.7 Merge to dev (#3766)

* [Update] 暂存，优化解决不了问题

* [Update] 待续（小白）

* [Update] 修改asset user

* [Update] 计划再次更改

* [Update] 修改asset user

* [Update] 暂存与喜爱

* [Update] Add id in

* [Update] 阶段性完成ops task该做

* [Update] 修改asset user api

* [Update] 修改asset user 任务，查看认证等

* [Update] 基本完成asset user改造

* [Update] dynamic user only allow 1

* [Update] 修改asset user task

* [Update] 修改node admin user task api

* [Update] remove file header license

* [Update] 添加sftp root

* [Update] 暂存

* [Update] 暂存

* [Update] 修改翻译

* [Update] 修改系统用户改为同名后，用户名改为空

* [Update] 基本完成CAS调研

* [Update] 支持cas server

* [Update] 支持cas server

* [Update] 添加requirements

* [Update] 为方便调试添加mysql ipython到包中

* [Update] 添加huaweiyun翻译

* [Update] 增加下载session 录像

* [Update] 只有第一次通知replay离线的使用方法

* [Update] 暂存一下

* [Bugfix] 获取系统用户信息报错

* [Bugfix] 修改system user info

* [Update] 改成清理10天git status

* [Update] 修改celery日志保留时间

* [Update]修复部分pip包依赖的版本不兼容问题 (#3672)

* [Update] 修复用户更新页面会清空用户public_key的问题

* Fix broken dependencies

Co-authored-by: BaiJiangJie <32935519+BaiJiangJie@users.noreply.github.com>

* [Update] 修改获取系统用户auth info

* [Update] Remove log

* [Bugfix] 修复sftp home设置的bug

* [Update] 授权的系统用户添加sftp root

* [Update] 修改系统用户关联的用户

* [Update] 修改placeholder

* [Update] 优化获取授权的系统用户

* [Update] 修改tasks

* [Update] tree service update

* [Update] 暂存

* [Update] 基本完成用户授权树和资产树改造

* [Update] Dashbaord perf

* [update] Add huawei cloud sdk requirements

* [Updte] 优化dashboard页面

* [Update] system user auth info 添加id

* [Update] 修改系统用户serializer

* [Update] 优化api

* [Update] LDAP Test Util (#3720)

* [Update] LDAPTestUtil 1

* [Update] LDAPTestUtil 2

* [Update] LDAPTestUtil 3

* [Update] LDAPTestUtil 4

* [Update] LDAPTestUtil 5

* [Update] LDAPTestUtil 6

* [Update] LDAPTestUtil 7

* [Update] session 已添加is success，并且添加display serializer

* [Bugfix] 修复无法删除空节点的bug

* [Update] 命令记录分组织显示

* [Update] Session is_success 添加迁移文件

* [Update] 批量命令添加org_id

* [Update] 修复一些文案，修改不绑定MFA，不能ssh登录

* [Update] 修改replay api, 返回session信息

* [Update] 解决无效es导致访问命令记录页面失败的问题

* [Update] 拆分profile view

* [Update] 修改一个翻译

* [Update] 修改aysnc api框架

* [Update] 命令列表添加risk level

* [Update] 完成录像打包下载

* [Update] 更改登陆otp页面

* [Update] 修改command 存储redis_level

* [Update] 修改翻译

* [Update] 修改系统用户的用户列表字段

* [Update] 使用新logo和统一Jumpserver为JumpServer

* [Update] 优化cloud task

* [Update] 统一period task

* [Update] 统一period form serializer字段

* [Update] 修改period task

* [Update] 修改资产网关信息

* [Update] 用户授权资产树资产信息添加domain

* [Update] 修改翻译

* [Update] 测试可连接性

* 1.5.7 bai (#3764)

* [Update] 修复index页面Bug；修复测试资产用户可连接性问题；

* [Update] 修改测试资产用户可连接

* [Bugfix] 修复backends问题

* [Update] 修改marksafe依赖版本

* [Update] 修改测试资产用户可连接性

* [Update] 修改检测服务器性能时获取percent值

* [Update] 更新依赖boto3=1.12.14

Co-authored-by: Yanzhe Lee <lee.yanzhe@yanzhe.org>
Co-authored-by: BaiJiangJie <32935519+BaiJiangJie@users.noreply.github.com>
Co-authored-by: Bai <bugatti_it@163.com>

apps/terminal/api/command.py

@@ -9,6 +9,7 @@ from rest_framework.response import Response
 from django.template import loader
 
 
+from orgs.utils import current_org
 from common.permissions import IsOrgAdminOrAppUser, IsOrgAuditor
 from common.utils import get_logger
 from ..backends import (
@@ -28,6 +29,22 @@ class CommandQueryMixin:
     ]
     default_days_ago = 5
 
+    @staticmethod
+    def get_org_id():
+        if current_org.is_default():
+            org_id = ''
+        else:
+            org_id = current_org.id
+        return org_id
+
+    def get_query_risk_level(self):
+        risk_level = self.request.query_params.get('risk_level')
+        if risk_level is None:
+            return None
+        if risk_level.isdigit():
+            return int(risk_level)
+        return None
+
     def get_queryset(self):
         # 解决访问 /docs/ 问题
         if hasattr(self, 'swagger_fake_view'):
@@ -38,7 +55,8 @@ class CommandQueryMixin:
         queryset = multi_command_storage.filter(
             date_from=date_from, date_to=date_to, input=q.get("input"),
             user=q.get("user"), asset=q.get("asset"),
-            system_user=q.get("system_user")
+            system_user=q.get("system_user"),
+            risk_level=self.get_query_risk_level(), org_id=self.get_org_id(),
         )
         return queryset
 

apps/terminal/api/session.py

@@ -1,21 +1,18 @@
 # -*- coding: utf-8 -*-
 #
-import os
-
-from django.shortcuts import get_object_or_404
+from django.shortcuts import get_object_or_404, reverse
 from django.core.files.storage import default_storage
-from django.http import HttpResponseNotFound
-from django.conf import settings
 from rest_framework import viewsets
 from rest_framework.response import Response
-import jms_storage
 
 from common.utils import is_uuid, get_logger
+from common.mixins.api import AsyncApiMixin
 from common.permissions import IsOrgAdminOrAppUser, IsOrgAuditor
 from common.drf.filters import DatetimeRangeFilter
 from orgs.mixins.api import OrgBulkModelViewSet
+from ..utils import find_session_replay_local, download_session_replay
 from ..hands import SystemUser
-from ..models import Session, ReplayStorage
+from ..models import Session
 from .. import serializers
 
 
@@ -25,7 +22,10 @@ logger = get_logger(__name__)
 
 class SessionViewSet(OrgBulkModelViewSet):
     model = Session
-    serializer_class = serializers.SessionSerializer
+    serializer_classes = {
+        'default': serializers.SessionSerializer,
+        'display': serializers.SessionDisplaySerializer,
+    }
     permission_classes = (IsOrgAdminOrAppUser, )
     filterset_fields = [
         "user", "asset", "system_user", "remote_addr",
@@ -59,10 +59,11 @@ class SessionViewSet(OrgBulkModelViewSet):
         return super().get_permissions()
 
 
-class SessionReplayViewSet(viewsets.ViewSet):
+class SessionReplayViewSet(AsyncApiMixin, viewsets.ViewSet):
     serializer_class = serializers.ReplaySerializer
     permission_classes = (IsOrgAdminOrAppUser | IsOrgAuditor,)
     session = None
+    download_cache_key = "SESSION_REPLAY_DOWNLOAD_{}"
 
     def create(self, request, *args, **kwargs):
         session_id = kwargs.get('pk')
@@ -83,46 +84,36 @@ class SessionReplayViewSet(viewsets.ViewSet):
             logger.error(msg)
             return Response({'msg': serializer.errors}, status=401)
 
-    def retrieve(self, request, *args, **kwargs):
-        session_id = kwargs.get('pk')
-        session = get_object_or_404(Session, id=session_id)
-
+    @staticmethod
+    def get_replay_data(session, url):
         tp = 'json'
         if session.protocol in ('rdp', 'vnc'):
             tp = 'guacamole'
 
-        data = {'type': tp, 'src': ''}
-
-        # 新版本和老版本的文件后缀不同
-        session_path = session.get_rel_replay_path()  # 存在外部存储上的路径
-        local_path = session.get_local_path()
-        local_path_v1 = session.get_local_path(version=1)
-
-        # 去default storage中查找
-        for _local_path in (local_path, local_path_v1, session_path):
-            if default_storage.exists(_local_path):
-                url = default_storage.url(_local_path)
-                data['src'] = url
-                return Response(data)
-
-        replay_storages = ReplayStorage.objects.all()
-        configs = {
-            storage.name: storage.config
-            for storage in replay_storages
-            if not storage.in_defaults()
+        download_url = reverse('terminal:session-replay-download', kwargs={'pk': session.id})
+        data = {
+            'type': tp, 'src': url,
+            'user': session.user, 'asset': session.asset,
+            'system_user': session.system_user,
+            'date_start': session.date_start,
+            'date_end': session.date_end,
+            'download_url': download_url,
         }
-        if not configs:
-            return HttpResponseNotFound()
+        return data
 
-        target_path = os.path.join(default_storage.base_location, local_path)   # 保存到storage的路径
-        target_dir = os.path.dirname(target_path)
-        if not os.path.isdir(target_dir):
-            os.makedirs(target_dir, exist_ok=True)
-        storage = jms_storage.get_multi_object_storage(configs)
-        ok, err = storage.download(session_path, target_path)
-        if not ok:
-            logger.error("Failed download replay file: {}".format(err))
-            return HttpResponseNotFound()
-        data['src'] = default_storage.url(local_path)
+    def is_need_async(self):
+        if self.action != 'retrieve':
+            return False
+        return True
+
+    def retrieve(self, request, *args, **kwargs):
+        session_id = kwargs.get('pk')
+        session = get_object_or_404(Session, id=session_id)
+        local_path, url = find_session_replay_local(session)
+
+        if not local_path:
+            local_path, url = download_session_replay(session)
+            if not local_path:
+                return Response({"error": url})
+        data = self.get_replay_data(session, url)
         return Response(data)
-