Compare commits

...

83 Commits
v2.8.1 ... v2.9

Author SHA1 Message Date
ibuler
ce2bbf08e7 fix: 修复周期监测任务配置的bug 2021-05-21 10:35:19 +08:00
老广
58c00ca09d Merge pull request #6050 from jumpserver/pr@v2.9@fix_expire_caches
fix: 添加启动失效缓存
2021-04-27 03:12:31 -05:00
xinwen
1e35fee1c9 fix: 添加启动失效缓存 2021-04-27 08:11:10 +00:00
Bai
87189412fe fix: 修改ACL提示支持的协议为: ssh、telnet 2021-04-23 16:35:26 +08:00
Bai
471cb45535 perf: 修改Latest version 2021-04-22 18:52:38 +08:00
Bai
11b0aa3b12 fix: 修复操作应用/应用授权/acl等未记录日志的问题2 2021-04-20 16:47:08 +08:00
ibuler
4b1b63f7b8 fix: 修复i18n一个翻译问题 2021-04-20 13:09:44 +08:00
Bai
512534715b fix: 修复操作应用/应用授权/acl等未记录日志的问题 2021-04-20 00:07:34 -05:00
ibuler
761ff5091a fix(task): 修复推送过期的问题 2021-04-19 22:42:30 -05:00
ibuler
87894df126 fix: 修复创建的系统用户很快过期的问题 2021-04-19 17:01:18 +08:00
Jiangjie.Bai
d042de7b09 Merge pull request #5972 from jumpserver/dev
v2.9.0 发版
2021-04-15 21:02:28 +08:00
ibuler
5e6e97c822 perf: 优化推送系统用户,设置有效期 2021-04-15 19:25:58 +08:00
xinwen
f146873501 fix: key=0 修改到 key=1 时 parent_key 没有更新 2021-04-15 01:33:00 -05:00
Jiangjie.Bai
35dfdf831a Merge pull request #5965 from jumpserver/dev
v2.9.0 rc3
2021-04-14 18:43:45 +08:00
xinwen
2b31cb2806 fix: 命令记录导出适配 ES 2021-04-14 05:02:50 -05:00
xinwen
e43ffa7994 fix: 远程应用显示名称 2021-04-14 04:56:59 -05:00
ibuler
b0a9a83231 fix(terminal): 修复终端列表看到的在线会话数量不对的bug 2021-04-14 16:41:57 +08:00
xinwen
7da14571ac fix: 请求 token 接口,登录类型没内容 2021-04-14 03:14:10 -05:00
xinwen
73b67da4c0 fix: 修复 acl 一些翻译 2021-04-14 03:13:18 -05:00
Jiangjie.Bai
4bf2371cf0 Merge pull request #5952 from jumpserver/dev
v2.9.0 rc2
2021-04-13 19:19:43 +08:00
Jiangjie.Bai
075cbc497b Merge pull request #5953 from jumpserver/pr@dev@dev_merge
chore(merge): 合并
2021-04-13 19:16:54 +08:00
ibuler
1a0d9a20f9 chore(merge): 合并 2021-04-13 18:54:08 +08:00
ibuler
fdb8416cac fix: 修复组件在线会话数量不对的问题 2021-04-13 05:49:03 -05:00
ibuler
e2d5b69510 perf: 优化健康监测,并添加 health check 的 key 2021-04-13 05:48:35 -05:00
xinwen
9944474ba0 fix: settings 订阅不稳定 2021-04-13 05:40:04 -05:00
xinwen
ce6b9de07c fix: ES 自动创建索引 2021-04-13 04:44:44 -05:00
xinwen
b97759687d fix: 邀请用没有触发信号 2021-04-13 04:23:29 -05:00
xinwen
68b6236de2 fix: SSO 登录日志 2021-04-12 04:48:17 -05:00
xinwen
6616374c30 fix: subscribe_settings_change 2021-04-12 04:45:30 -05:00
xinwen
682f6b2fb9 fix: 资产节点关系变化时也要清空 root 组织的 node_assets_mapping 2021-04-12 04:44:13 -05:00
xinwen
a2e3979916 fix: org_mapping 保护订阅线程 2021-04-12 04:42:48 -05:00
xinwen
f11d3c1cf2 fix: 过期用户登录提示无效 2021-04-09 02:03:35 -05:00
xinwen
f0bad5f107 fix: 登录页面测试 cookie 失败 2021-04-09 01:46:21 -05:00
ibuler
ad3bc72dfb fix(terminal): 修复session id 长度误写为 35 的bug 2021-04-08 19:23:36 +08:00
xinwen
de9c69843d fix: 登录日志 user_agent 过长 2021-04-08 19:23:36 +08:00
xinwen
d2678e2a43 refactor: 移动 PermissionsMixin 位置 2021-04-08 19:23:36 +08:00
xinwen
632ea87f07 feat: MFA 登录次数限制 2021-04-08 19:23:36 +08:00
fit2bot
4e7e1d5e15 style: 优化全局组织设置相关代码 (#5921)
* feat:支持配置全局组织的显示名称

* style: 优化全局组织设置相关代码

Co-authored-by: liubo <liubo@fit2cloud.com>
2021-04-08 19:23:36 +08:00
liuboF2c
1ac8537a34 feat:支持配置全局组织的显示名称 (#5919)
Co-authored-by: liubo <liubo@fit2cloud.com>
2021-04-08 19:23:36 +08:00
fit2bot
dcaa798c2e perf: csv upload (#5894)
perf: 修改翻译

Co-authored-by: ibuler <ibuler@qq.com>
2021-04-08 19:23:36 +08:00
xinwen
8da4027e32 fix: 授权资产列表 platform 应该显示名称 2021-04-08 19:23:36 +08:00
xinwen
32e2d19553 fix: 改密计划关掉周期执行再打开,任务不再执行 2021-04-08 19:23:36 +08:00
xinwen
48d1eecc08 fix: 修正 key 为 0 的节点 2021-04-08 19:23:36 +08:00
xinwen
0ab88ce754 fix: 访问 tokens 接口更新用户最后登录时间 2021-04-08 19:23:36 +08:00
xinwen
bee5500425 fix: 创建节点的时候加锁,可以并发调用 2021-04-08 19:23:36 +08:00
xinwen
7c03af7668 feat: 资产授权支持按名称模糊搜索 2021-04-08 19:23:36 +08:00
xinwen
7a61a671a2 fix: 管理用户输入带密码的秘钥报错 2021-04-08 19:23:36 +08:00
Bai
4a1fc0e2ac fix: 修复NodeChildrenAddAPI不支持patch方法的问题 2021-04-08 19:23:36 +08:00
ibuler
1e5e87e62a perf: 优化acl提示 2021-04-08 19:23:36 +08:00
ibuler
96c3b81383 perf: upgrade requirements version 2021-04-08 19:23:36 +08:00
xinwen
297fedeffa fix: Default 组织下出现 app user 2021-04-08 19:23:36 +08:00
ibuler
9cd5675209 perf: 修改terminal statuts
perf: 优化status api

perf: 优化 status api

perf: 修改sesion参数

perf: 修改migrations

perf: 优化数据结构

perf: 修改保留日志

perf: 优化之前的一个写法
2021-04-08 19:23:36 +08:00
xinwen
a5179d1596 feat: 增加 es 忽略 https 证书验证 2021-04-08 19:23:36 +08:00
Bai
c2463fe573 perf: Session Login from 添加 RDP Terminal 类型 2021-04-08 19:23:36 +08:00
xinwen
2f8042141c fix: 授权树节点排序 2021-04-08 19:23:36 +08:00
ibuler
06a4e0d395 perf: 修改表结构迁移,增加rdp terminal 2021-04-08 19:23:36 +08:00
xinwen
bb9d92fd7e perf: delete_test_cookie 2021-04-08 19:23:36 +08:00
ibuler
749f9d3f81 fix(terminal): 修复session id 长度误写为 35 的bug 2021-04-08 17:39:29 +08:00
xinwen
03ad7777d0 fix: 登录日志 user_agent 过长 2021-04-08 04:39:12 -05:00
xinwen
7e4f20f443 refactor: 移动 PermissionsMixin 位置 2021-04-08 02:15:02 -05:00
xinwen
607b7fd29f feat: MFA 登录次数限制 2021-04-08 01:46:36 -05:00
fit2bot
8895763ab4 style: 优化全局组织设置相关代码 (#5921)
* feat:支持配置全局组织的显示名称

* style: 优化全局组织设置相关代码

Co-authored-by: liubo <liubo@fit2cloud.com>
2021-04-08 14:18:53 +08:00
liuboF2c
8b1e202e68 feat:支持配置全局组织的显示名称 (#5919)
Co-authored-by: liubo <liubo@fit2cloud.com>
2021-04-08 13:55:58 +08:00
fit2bot
32fe8f674c perf: csv upload (#5894)
perf: 修改翻译

Co-authored-by: ibuler <ibuler@qq.com>
2021-04-08 10:11:46 +08:00
xinwen
b4ef7bef55 fix: 授权资产列表 platform 应该显示名称 2021-04-08 10:10:32 +08:00
xinwen
31982c6547 fix: 改密计划关掉周期执行再打开,任务不再执行 2021-04-07 18:38:45 +08:00
xinwen
67d3b63c6d fix: 修正 key 为 0 的节点 2021-04-07 11:11:23 +08:00
xinwen
f34fb5d9d5 fix: 访问 tokens 接口更新用户最后登录时间 2021-04-07 10:45:34 +08:00
xinwen
3ec78ff9be fix: 创建节点的时候加锁,可以并发调用 2021-04-07 10:29:19 +08:00
xinwen
f361621ab5 feat: 资产授权支持按名称模糊搜索 2021-04-07 10:28:14 +08:00
xinwen
cd9587f68e fix: 管理用户输入带密码的秘钥报错 2021-04-06 19:46:16 +08:00
Bai
2ff01a4bb3 fix: 修复NodeChildrenAddAPI不支持patch方法的问题 2021-04-01 10:55:21 +08:00
ibuler
06ed358fbc perf: 优化acl提示 2021-03-30 10:36:41 +08:00
ibuler
3e11249e8c perf: upgrade requirements version 2021-03-30 10:25:30 +08:00
xinwen
6b5435b768 fix: Default 组织下出现 app user 2021-03-30 10:24:25 +08:00
ibuler
7d5a13de38 perf: 修改terminal statuts
perf: 优化status api

perf: 优化 status api

perf: 修改sesion参数

perf: 修改migrations

perf: 优化数据结构

perf: 修改保留日志

perf: 优化之前的一个写法
2021-03-29 19:21:32 +08:00
xinwen
07bd44990b feat: 增加 es 忽略 https 证书验证 2021-03-29 15:23:33 +08:00
noon
e4938ffc85 Update README_EN.md (#5856)
* Update README_EN.md

Translate parts of the README.md

* Update README_EN.md

* Update README_EN.md

change the word PAM to Bastion host

* Update README_EN.md

* Update README_EN.md

Clip the bug part to JumpServer 远程执行漏洞 2021-01-15
2021-03-27 20:14:45 +08:00
Bai
85d226eb07 perf: Session Login from 添加 RDP Terminal 类型 2021-03-26 10:38:48 +08:00
xinwen
c9a9ca7923 fix: 授权树节点排序 2021-03-24 10:23:45 +08:00
ibuler
306f7a08d1 perf: 修改表结构迁移,增加rdp terminal 2021-03-24 10:14:59 +08:00
老广
b86f9ac871 Update README.md 2021-03-23 15:47:19 +08:00
xinwen
2562386fe0 perf: delete_test_cookie 2021-03-23 15:27:06 +08:00
78 changed files with 1626 additions and 875 deletions

View File

@@ -1,7 +1,7 @@
# JumpServer 多云环境下更好用的堡垒机
[![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
[![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
[![License](https://shields.io/github/license/jumpserver/jumpserver)](https://www.gnu.org/licenses/old-licenses/gpl-2.0.html)
[![Release Downloads](https://shields.io/github/downloads/jumpserver/jumpserver/total)](https://github.com/jumpserver/jumpserver/releases)
[![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
- [ENGLISH](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)

View File

@@ -1,143 +1,245 @@
## Jumpserver
# Jumpserver - The Bastion Host for Multi-Cloud Environment
[![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
[![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
[![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
----
## CRITICAL BUG WARNING
- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
Recently we have found a critical bug for remote execution vulnerability which leads to pre-auth and info leak, please fix it as soon as possible.
Thanks for **reactivity from Alibaba Hackerone bug bounty program** report us this bug
**Vulnerable version:**
```
< v2.6.2
< v2.5.4
< v2.4.5
= v1.5.9
>= v1.5.3
```
**Safe and Stable version:**
```
>= v2.6.2
>= v2.5.4
>= v2.4.5
= v1.5.9 version tag didn't change
< v1.5.3
```
**Bug Fix Solution:**
Upgrade to the latest version or the version mentioned above
**Temporary Solution (upgrade asap):**
Modify the Nginx config file and disable the vulnerable api listed below
```
/api/v1/authentication/connection-token/
/api/v1/users/connection-token/
```
Path to Nginx config file
```
# Previous Community version
/etc/nginx/conf.d/jumpserver.conf
# Previous Enterprise version
jumpserver-release/nginx/http_server.conf
# Latest version
jumpserver-release/compose/config_static/http_server.conf
```
Changes in Nginx config file
```
### Put the following code on top of location server, or before /api and /
location /api/v1/authentication/connection-token/ {
return 403;
}
location /api/v1/users/connection-token/ {
return 403;
}
### End right here
location /api/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://core:8080;
}
...
```
Save the file and restart Nginx
```
docker deployment:
$ docker restart jms_nginx
rpm or other deployment:
$ systemctl restart nginx
```
**Bug Fix Verification**
```
# Download the following script to check if it is fixed
$ wget https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_bug_check.sh
# Run the code to verify it
$ bash jms_bug_check.sh demo.jumpserver.org
漏洞已修复 (It means the bug is fixed)
漏洞未修复 (It means the bug is not fixed and the system is still vulnerable)
```
**Attack Simulation**
Go to the logs directory which should contain gunicorn.log file. Then download the "attack" script and execute it
```
$ pwd
/opt/jumpserver/core/logs
$ ls gunicorn.log
gunicorn.log
$ wget 'https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_check_attack.sh'
$ bash jms_check_attack.sh
系统未被入侵 (It means the system is safe)
系统已被入侵 (It means the system is being attacked)
```
|![notification](https://raw.githubusercontent.com/goharbor/website/master/docs/img/readme/bell-outline-badged.svg)Security Notice|
|------------------|
|On 15th January 2021, JumpServer found a critical bug for remote execution vulnerability. Please fix it asap! [For more detail](https://github.com/jumpserver/jumpserver/issues/5533) Thanks for **reactivity of Alibaba Hackerone bug bounty program** report use the bug|
--------------------------
----
- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
Jumpserver is the world's first open-source PAM (Privileged Access Management System) and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
Jumpserver is the world's first open-source Bastion Host and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
Jumpserver uses Python / Django for development, follows Web 2.0 specifications, and is equipped with an industry-leading Web Terminal solution that provides a beautiful user interface and great user experience
Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple cross-regional areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
Change the world, starting from little things.
Change the world by taking every little step
----
### Advantages
### Features
- Open Source: huge transparency and free to access with quick installation process.
- Distributed: support large-scale concurrent access with ease.
- No Plugin required: all you need is a browser, the ultimate Web Terminal experience.
- Multi-Cloud supported: a unified system to manage assets on different clouds at the same time
- Cloud storage: audit records are stored in the cloud. Data lost no more!
- Multi-Tenant system: multiple subsidiary companies or departments access the same system simultaneously.
- Many applications supported: link to databases, windows remote applications, and Kubernetes cluster, etc.
![Jumpserver 功能](https://jumpserver-release.oss-cn-hangzhou.aliyuncs.com/Jumpserver148.jpeg "Jumpserver 功能")
## Features List
<table>
<tr>
<td rowspan="8">Authentication</td>
<td rowspan="5">Login</td>
<td>Unified way to access and authenticate resources</td>
</tr>
<tr>
<td>LDAP/AD Authentication</td>
</tr>
<tr>
<td>RADIUS Authentication</td>
</tr>
<tr>
<td>OpenID AuthenticationSingle Sign-On</td>
</tr>
<tr>
<td>CAS Authentication Single Sign-On</td>
</tr>
<tr>
<td rowspan="2">MFA (Multi-Factor Authentication)</td>
<td>Use Google Authenticator for MFA</td>
</tr>
<tr>
<td>RADIUS (Remote Authentication Dial In User Service)</td>
</tr>
<tr>
<td>Login Supervision</td>
<td>Any users login behavior is supervised and controlled by the administrator:small_orange_diamond:</td>
</tr>
<tr>
<td rowspan="11">Accounting</td>
<td rowspan="2">Centralized Accounts Management</td>
<td>Admin Users management</td>
</tr>
<tr>
<td>System Users management</td>
</tr>
<tr>
<td rowspan="4">Unified Password Management</td>
<td>Asset password custody (a matrix storing all asset password with dense security)</td>
</tr>
<tr>
<td>Auto-generated passwords</td>
</tr>
<tr>
<td>Automatic password handling (auto login assets)</td>
</tr>
<tr>
<td>Password expiration settings</td>
</tr>
<tr>
<td rowspan="2">Password change Schedular</td>
<td>Support regular batch Linux/Windows assets password changing:small_orange_diamond:</td>
</tr>
<tr>
<td>Implement multiple password strategies:small_orange_diamond:</td>
</tr>
<tr>
<td>Multi-Cloud Management</td>
<td>Automatically manage private cloud and public cloud assets in a unified platform :small_orange_diamond:</td>
</tr>
<tr>
<td>Users Acquisition </td>
<td>Create regular custom tasks to collect system users in selected assets to identify and track the privileges ownership:small_orange_diamond:</td>
</tr>
<tr>
<td>Password Vault </td>
<td>Unified operations to check, update, and test system user password to prevent stealing or unauthorised sharing of passwords:small_orange_diamond:</td>
</tr>
<tr>
<td rowspan="15">Authorization</td>
<td>Multi-Dimensional</td>
<td>Granting users or user groups to access assets, asset nodes, or applications through system users. Providing precise access control to different roles of users</td>
</tr>
<tr>
<td rowspan="4">Assets</td>
<td>Assets are arranged and displayed in a tree structure </td>
</tr>
<tr>
<td>Assets and Nodes have immense flexibility for authorizing</td>
</tr>
<tr>
<td>Assets in nodes inherit authorization automatically</td>
</tr>
<tr>
<td>child nodes automatically inherit authorization from parent nodes</td>
</tr>
<tr>
<td rowspan="2">Application</td>
<td>Provides granular access control for privileged users on application level to protect from unauthorized access and unintentional errors</td>
</tr>
<tr>
<td>Database applications (MySQL, Oracle, PostgreSQL, MariaDB, etc.) and Remote App:small_orange_diamond: </td>
</tr>
<tr>
<td>Actions</td>
<td>Deeper restriction on the control of file upload, download and connection actions of authorized assets. Control the permission of clipboard copy/paste (from outer terminal to current asset)</td>
</tr>
<tr>
<td>Time Bound</td>
<td>Sharply limited the available (accessible) time for account access to the authorized resources to reduce the risk and attack surface drastically</td>
</tr>
<tr>
<td>Privileged Assignment</td>
<td>Assign the denied/allowed command lists to different system users as privilege elevation, with the latter taking the form of allowing particular commands to be run with a higher level of privileges. (Minimize insider threat)</td>
</tr>
<tr>
<td>Command Filtering</td>
<td>Creating list of restriction commands that you would like to assign to different authorized system users for filtering purpose</td>
</tr>
<tr>
<td>File Transfer and Management</td>
<td>Support SFTP file upload/download</td>
</tr>
<tr>
<td>File Management</td>
<td>Provide a Web UI for SFTP file management</td>
</tr>
<tr>
<td>Workflow Management</td>
<td>Manage user login confirmation requests and assets or applications authorization requests for Just-In-Time Privileges functionality:small_orange_diamond:</td>
</tr>
<tr>
<td>Group Management </td>
<td>Establishing a multi-tenant ecosystem that able authority isolation to keep malicious actors away from sensitive administrative backends:small_orange_diamond:</td>
</tr>
<tr>
<td rowspan="8">Auditing</td>
<td>Operations</td>
<td>Auditing user operation behaviors for any access or usage of given privileged accounts</td>
</tr>
<tr>
<td rowspan="2">Session</td>
<td>Support real-time session audit</td>
</tr>
<tr>
<td>Full history of all previous session audits</td>
</tr>
<tr>
<td rowspan="3">Video</td>
<td>Complete session audit and playback recordings on assets operation (Linux, Windows)</td>
</tr>
<tr>
<td>Full recordings of RemoteApp, MySQL, and Kubernetes:small_orange_diamond:</td>
</tr>
<tr>
<td>Supports uploading recordings to public clouds</td>
</tr>
<tr>
<td>Command</td>
<td>Command auditing on assets and applications operation. Send warning alerts when executing illegal commands</td>
</tr>
<tr>
<td>File Transfer</td>
<td>Full recordings of file upload and download</td>
</tr>
<tr>
<td rowspan="20">Database</td>
<td rowspan="2">How to connect</td>
<td>Command line</td>
</tr>
<tr>
<td>Built-in Web UI:small_orange_diamond:</td>
</tr>
<tr>
<td rowspan="4">Supported Database</td>
<td>MySQL</td>
</tr>
<tr>
<td>Oracle :small_orange_diamond:</td>
</tr>
<tr>
<td>MariaDB :small_orange_diamond:</td>
</tr>
<tr>
<td>PostgreSQL :small_orange_diamond:</td>
</tr>
<tr>
<td rowspan="6">Feature Highlights</td>
<td>Syntax highlights</td>
</tr>
<tr>
<td>Prettier SQL formmating</td>
</tr>
<tr>
<td>Support Shortcuts</td>
</tr>
<tr>
<td>Support selected SQL statements</td>
</tr>
<tr>
<td>SQL commands history query</td>
</tr>
<tr>
<td>Support page creation: DB, TABLE</td>
</tr>
<tr>
<td rowspan="2">Session Auditing</td>
<td>Full records of command</td>
</tr>
<tr>
<td>Playback videos</td>
</tr>
</table>
**Note**: Rows with :small_orange_diamond: at the end of the sentence means that it is X-PACK features exclusive ([Apply for X-PACK Trial](https://jinshuju.net/f/kyOYpi))
### Start
@@ -162,6 +264,50 @@ We provide the SDK for your other systems to quickly interact with the Jumpserve
- [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver other components use this SDK to complete the interaction.
- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) Thanks to 恺珺 for providing his Java SDK vesrion.
## JumpServer Component Projects
- [Lina](https://github.com/jumpserver/lina) JumpServer Web UI
- [Luna](https://github.com/jumpserver/luna) JumpServer Web Terminal
- [KoKo](https://github.com/jumpserver/koko) JumpServer Character protocaol Connector, replace original Python Version [Coco](https://github.com/jumpserver/coco)
- [Guacamole](https://github.com/jumpserver/docker-guacamole) JumpServer Graphics protocol Connectorrely on [Apache Guacamole](https://guacamole.apache.org/)
## Contribution
If you have any good ideas or helping us to fix bugs, please submit a Pull Request and accept our thanks :)
Thanks to the following contributors for making JumpServer better everyday!
<a href="https://github.com/jumpserver/jumpserver/graphs/contributors">
<img src="https://contrib.rocks/image?repo=jumpserver/jumpserver" />
</a>
## Thanks to
- [Apache Guacamole](https://guacamole.apache.org/) Web page connection RDP, SSH, VNC protocol equipment. JumpServer graphical connection dependent.
- [OmniDB](https://omnidb.org/) Web page connection to databases. JumpServer Web database dependent.
## JumpServer Enterprise Version
- [Apply for it](https://jinshuju.net/f/kyOYpi)
## Case Study
- [JumpServer 堡垒机护航顺丰科技超大规模资产安全运维](https://blog.fit2cloud.com/?p=1147)
- [JumpServer 堡垒机让“大智慧”的混合 IT 运维更智慧](https://blog.fit2cloud.com/?p=882)
- [携程 JumpServer 堡垒机部署与运营实战](https://blog.fit2cloud.com/?p=851)
- [小红书的JumpServer堡垒机大规模资产跨版本迁移之路](https://blog.fit2cloud.com/?p=516)
- [JumpServer堡垒机助力中手游提升多云环境下安全运维能力](https://blog.fit2cloud.com/?p=732)
- [中通快递JumpServer主机安全运维实践](https://blog.fit2cloud.com/?p=708)
- [东方明珠JumpServer高效管控异构化、分布式云端资产](https://blog.fit2cloud.com/?p=687)
- [江苏农信JumpServer堡垒机助力行业云安全运维](https://blog.fit2cloud.com/?p=666)。
## For safety instructions
JumpServer is a security product. Please refer to [Basic Security Recommendations](https://docs.jumpserver.org/zh/master/install/install_security/) for deployment and installation.
If you find a security problem, please contact us directly
- ibuler@fit2cloud.com
- support@fit2cloud.com
- 400-052-0755
### License & Copyright
Copyright (c) 2014-2019 Beijing Duizhan Tech, Inc., All rights reserved.

View File

@@ -1,9 +0,0 @@
from django.utils.translation import ugettext as _
common_help_text = _('Format for comma-delimited string, with * indicating a match all. ')
ip_group_help_text = common_help_text + _(
'Such as: '
'192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
)

View File

@@ -33,6 +33,9 @@ class LoginACL(BaseACL):
class Meta:
ordering = ('priority', '-date_updated', 'name')
def __str__(self):
return self.name
@property
def action_reject(self):
return self.action == self.ActionChoices.reject

View File

@@ -38,6 +38,9 @@ class LoginAssetACL(BaseACL, OrgModelMixin):
unique_together = ('name', 'org_id')
ordering = ('priority', '-date_updated', 'name')
def __str__(self):
return self.name
@classmethod
def filter(cls, user, asset, system_user, action):
queryset = cls.objects.filter(action=action)

View File

@@ -4,7 +4,6 @@ from common.drf.serializers import BulkModelSerializer
from orgs.utils import current_org
from ..models import LoginACL
from ..utils import is_ip_address, is_ip_network, is_ip_segment
from .. import const
__all__ = ['LoginACLSerializer', ]
@@ -21,8 +20,14 @@ def ip_group_child_validator(ip_group_child):
class LoginACLSerializer(BulkModelSerializer):
ip_group_help_text = _(
'Format for comma-delimited string, with * indicating a match all. '
'Such as: '
'192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
)
ip_group = serializers.ListField(
default=['*'], label=_('IP'), help_text=const.ip_group_help_text,
default=['*'], label=_('IP'), help_text=ip_group_help_text,
child=serializers.CharField(max_length=1024, validators=[ip_group_child_validator])
)
user_display = serializers.ReadOnlyField(source='user.name', label=_('User'))

View File

@@ -1,46 +1,60 @@
from rest_framework import serializers
from django.utils.translation import ugettext as _
from django.utils.translation import ugettext_lazy as _
from orgs.mixins.serializers import BulkOrgResourceModelSerializer
from assets.models import SystemUser
from acls import models
from orgs.models import Organization
from .. import const
__all__ = ['LoginAssetACLSerializer']
common_help_text = _('Format for comma-delimited string, with * indicating a match all. ')
class LoginAssetACLUsersSerializer(serializers.Serializer):
username_group = serializers.ListField(
default=['*'], child=serializers.CharField(max_length=128), label=_('Username'),
help_text=const.common_help_text
help_text=common_help_text
)
class LoginAssetACLAssestsSerializer(serializers.Serializer):
ip_group_help_text = _(
'Format for comma-delimited string, with * indicating a match all. '
'Such as: '
'192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
'(Domain name support)'
)
ip_group = serializers.ListField(
default=['*'], child=serializers.CharField(max_length=1024), label=_('IP'),
help_text=const.ip_group_help_text + _('(Domain name support)')
help_text=ip_group_help_text
)
hostname_group = serializers.ListField(
default=['*'], child=serializers.CharField(max_length=128), label=_('Hostname'),
help_text=const.common_help_text
help_text=common_help_text
)
class LoginAssetACLSystemUsersSerializer(serializers.Serializer):
protocol_group_help_text = _(
'Format for comma-delimited string, with * indicating a match all. '
'Protocol options: {}'
)
name_group = serializers.ListField(
default=['*'], child=serializers.CharField(max_length=128), label=_('Name'),
help_text=const.common_help_text
help_text=common_help_text
)
username_group = serializers.ListField(
default=['*'], child=serializers.CharField(max_length=128), label=_('Username'),
help_text=const.common_help_text
help_text=common_help_text
)
protocol_group = serializers.ListField(
default=['*'], child=serializers.CharField(max_length=16), label=_('Protocol'),
help_text=const.common_help_text + _('Protocol options: {}').format(
', '.join(SystemUser.ASSET_CATEGORY_PROTOCOLS)
help_text=protocol_group_help_text.format(
', '.join([SystemUser.PROTOCOL_SSH, SystemUser.PROTOCOL_TELNET])
)
)

View File

@@ -44,8 +44,8 @@ class ApplicationSerializerMixin(serializers.Serializer):
class ApplicationSerializer(ApplicationSerializerMixin, BulkOrgResourceModelSerializer):
category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category'))
type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type'))
category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category(Display)'))
type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type(Dispaly)'))
class Meta:
model = models.Application

View File

@@ -28,6 +28,7 @@ from ..tasks import (
)
from .. import serializers
from .mixin import SerializeToTreeNodeMixin
from assets.locks import NodeAddChildrenLock
logger = get_logger(__file__)
@@ -114,15 +115,16 @@ class NodeChildrenApi(generics.ListCreateAPIView):
return super().initial(request, *args, **kwargs)
def perform_create(self, serializer):
data = serializer.validated_data
_id = data.get("id")
value = data.get("value")
if not value:
value = self.instance.get_next_child_preset_name()
node = self.instance.create_child(value=value, _id=_id)
# 避免查询 full value
node._full_value = node.value
serializer.instance = node
with NodeAddChildrenLock(self.instance):
data = serializer.validated_data
_id = data.get("id")
value = data.get("value")
if not value:
value = self.instance.get_next_child_preset_name()
node = self.instance.create_child(value=value, _id=_id)
# 避免查询 full value
node._full_value = node.value
serializer.instance = node
def get_object(self):
pk = self.kwargs.get('pk') or self.request.query_params.get('id')
@@ -221,7 +223,8 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
serializer_class = serializers.NodeAddChildrenSerializer
instance = None
def put(self, request, *args, **kwargs):
def update(self, request, *args, **kwargs):
""" 同时支持 put 和 patch 方法"""
instance = self.get_object()
node_ids = request.data.get("nodes")
children = Node.objects.filter(id__in=node_ids)

View File

@@ -1,5 +1,6 @@
from orgs.utils import current_org
from common.utils.lock import DistributedLock
from assets.models import Node
class NodeTreeUpdateLock(DistributedLock):
@@ -18,3 +19,11 @@ class NodeTreeUpdateLock(DistributedLock):
def __init__(self):
name = self.get_name()
super().__init__(name=name, release_on_transaction_commit=True, reentrant=True)
class NodeAddChildrenLock(DistributedLock):
name_template = 'assets.node.add_children.<org_id:{org_id}>'
def __init__(self, node: Node):
name = self.name_template.format(org_id=node.org_id)
super().__init__(name=name, release_on_transaction_commit=True)

View File

@@ -0,0 +1,61 @@
from django.db import migrations
from django.db.transaction import atomic
default_id = '00000000-0000-0000-0000-000000000002'
def change_key0_to_key1(apps, schema_editor):
from orgs.utils import set_current_org
# https://stackoverflow.com/questions/28777338/django-migrations-runpython-not-able-to-call-model-methods
Organization = apps.get_model('orgs', 'Organization')
Node = apps.get_model('assets', 'Node')
print()
org = Organization.objects.get(id=default_id)
set_current_org(org)
exists_0 = Node.objects.filter(key__startswith='0').exists()
if not exists_0:
print(f'--> Not exist key=0 nodes, do nothing.')
return
key_1_count = Node.objects.filter(key__startswith='1').count()
if key_1_count > 1:
print(f'--> Node key=1 have children, can`t just delete it. Please contact JumpServer team')
return
root_node = Node.objects.filter(key='1').first()
if root_node and root_node.assets.exists():
print(f'--> Node key=1 has assets, do nothing.')
return
with atomic():
if root_node:
print(f'--> Delete node key=1')
root_node.delete()
nodes_0 = Node.objects.filter(key__startswith='0')
for n in nodes_0:
old_key = n.key
key_list = n.key.split(':')
key_list[0] = '1'
new_key = ':'.join(key_list)
new_parent_key = ':'.join(key_list[:-1])
n.key = new_key
n.parent_key = new_parent_key
n.save()
print('--> Modify key ( {} > {} )'.format(old_key, new_key))
class Migration(migrations.Migration):
dependencies = [
('orgs', '0010_auto_20210219_1241'),
('assets', '0068_auto_20210312_1455'),
]
operations = [
migrations.RunPython(change_key0_to_key1)
]

View File

@@ -113,7 +113,7 @@ class AuthMixin:
if self.public_key:
public_key = self.public_key
elif self.private_key:
public_key = ssh_pubkey_gen(self.private_key, self.password)
public_key = ssh_pubkey_gen(private_key=self.private_key, password=self.password)
else:
return ''

View File

@@ -307,6 +307,15 @@ class NodeAllAssetsMappingMixin:
org_id = str(org_id)
cls.orgid_nodekey_assetsid_mapping.pop(org_id, None)
@classmethod
def expire_all_orgs_node_all_asset_ids_mapping_from_memory(cls):
orgs = Organization.objects.all()
org_ids = [str(org.id) for org in orgs]
org_ids.append(Organization.ROOT_ID)
for id in org_ids:
cls.expire_node_all_asset_ids_mapping_from_memory(id)
# get order: from memory -> (from cache -> to generate)
@classmethod
def get_node_all_asset_ids_mapping_from_cache_or_generate_to_cache(cls, org_id):

View File

@@ -13,6 +13,7 @@ from common.signals import django_ready
from common.utils.connection import RedisPubSub
from common.utils import get_logger
from assets.models import Asset, Node
from orgs.models import Organization
logger = get_logger(__file__)
@@ -36,13 +37,18 @@ node_assets_mapping_for_memory_pub_sub = NodeAssetsMappingForMemoryPubSub()
def expire_node_assets_mapping_for_memory(org_id):
# 所有进程清除(自己的 memory 数据)
org_id = str(org_id)
node_assets_mapping_for_memory_pub_sub.publish(org_id)
root_org_id = Organization.ROOT_ID
# 当前进程清除(cache 数据)
logger.debug(
"Expire node assets id mapping from cache of org={}, pid={}"
"".format(org_id, os.getpid())
)
Node.expire_node_all_asset_ids_mapping_from_cache(org_id)
Node.expire_node_all_asset_ids_mapping_from_cache(root_org_id)
node_assets_mapping_for_memory_pub_sub.publish(org_id)
node_assets_mapping_for_memory_pub_sub.publish(root_org_id)
@receiver(post_save, sender=Node)
@@ -73,16 +79,22 @@ def subscribe_node_assets_mapping_expire(sender, **kwargs):
logger.debug("Start subscribe for expire node assets id mapping from memory")
def keep_subscribe():
subscribe = node_assets_mapping_for_memory_pub_sub.subscribe()
for message in subscribe.listen():
if message["type"] != "message":
continue
org_id = message['data'].decode()
Node.expire_node_all_asset_ids_mapping_from_memory(org_id)
logger.debug(
"Expire node assets id mapping from memory of org={}, pid={}"
"".format(str(org_id), os.getpid())
)
while True:
try:
subscribe = node_assets_mapping_for_memory_pub_sub.subscribe()
for message in subscribe.listen():
if message["type"] != "message":
continue
org_id = message['data'].decode()
Node.expire_node_all_asset_ids_mapping_from_memory(org_id)
logger.debug(
"Expire node assets id mapping from memory of org={}, pid={}"
"".format(str(org_id), os.getpid())
)
except Exception as e:
logger.exception(f'subscribe_node_assets_mapping_expire: {e}')
Node.expire_all_orgs_node_all_asset_ids_mapping_from_memory()
t = threading.Thread(target=keep_subscribe)
t.daemon = True
t.start()

View File

@@ -56,6 +56,7 @@ def get_push_unixlike_system_user_tasks(system_user, username=None):
'shell': system_user.shell or Empty,
'state': 'present',
'home': system_user.home or Empty,
'expires': -1,
'groups': groups or Empty,
'comment': comment
}

View File

@@ -0,0 +1,18 @@
# Generated by Django 3.1 on 2021-04-14 06:43
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('audits', '0011_userloginlog_backend'),
]
operations = [
migrations.AlterField(
model_name='userloginlog',
name='type',
field=models.CharField(choices=[('W', 'Web'), ('T', 'Terminal'), ('U', 'Unknown')], max_length=2, verbose_name='Login type'),
),
]

View File

@@ -79,6 +79,7 @@ class UserLoginLog(models.Model):
LOGIN_TYPE_CHOICE = (
('W', 'Web'),
('T', 'Terminal'),
('U', 'Unknown'),
)
MFA_DISABLED = 0

View File

@@ -27,11 +27,23 @@ json_render = JSONRenderer()
MODELS_NEED_RECORD = (
'User', 'UserGroup', 'Asset', 'Node', 'AdminUser', 'SystemUser',
'Domain', 'Gateway', 'Organization', 'AssetPermission', 'CommandFilter',
'CommandFilterRule', 'License', 'Setting', 'Account', 'SyncInstanceTask',
'Platform', 'ChangeAuthPlan', 'GatherUserTask',
'RemoteApp', 'RemoteAppPermission', 'DatabaseApp', 'DatabaseAppPermission',
# users
'User', 'UserGroup',
# acls
'LoginACL', 'LoginAssetACL',
# assets
'Asset', 'Node', 'AdminUser', 'SystemUser', 'Domain', 'Gateway', 'CommandFilterRule',
'CommandFilter', 'Platform',
# applications
'Application',
# orgs
'Organization',
# settings
'Setting',
# perms
'AssetPermission', 'ApplicationPermission',
# xpack
'License', 'Account', 'SyncInstanceTask', 'ChangeAuthPlan', 'GatherUserTask',
)
@@ -44,6 +56,7 @@ class AuthBackendLabelMapping(LazyObject):
backend_label_mapping[backend] = source.label
backend_label_mapping[settings.AUTH_BACKEND_PUBKEY] = _('SSH Key')
backend_label_mapping[settings.AUTH_BACKEND_MODEL] = _('Password')
backend_label_mapping[settings.AUTH_BACKEND_SSO] = _('SSO')
return backend_label_mapping
def _setup(self):
@@ -145,7 +158,7 @@ def generate_data(username, request):
user_agent = request.META.get('HTTP_USER_AGENT', '')
login_ip = get_request_ip(request) or '0.0.0.0'
if isinstance(request, Request):
login_type = request.META.get('HTTP_X_JMS_LOGIN_TYPE', '')
login_type = request.META.get('HTTP_X_JMS_LOGIN_TYPE', 'U')
else:
login_type = 'W'
@@ -153,7 +166,7 @@ def generate_data(username, request):
'username': username,
'ip': login_ip,
'type': login_type,
'user_agent': user_agent,
'user_agent': user_agent[0:254],
'datetime': timezone.now(),
'backend': get_login_backend(request)
}

View File

@@ -29,7 +29,7 @@ class MFAChallengeApi(AuthMixin, CreateAPIView):
if not valid:
self.request.session['auth_mfa'] = ''
raise errors.MFAFailedError(
username=user.username, request=self.request
username=user.username, request=self.request, ip=self.get_request_ip()
)
else:
self.request.session['auth_mfa'] = '1'

View File

@@ -6,9 +6,7 @@ from django.conf import settings
from common.exceptions import JMSException
from .signals import post_auth_failed
from users.utils import (
increase_login_failed_count, get_login_failed_count
)
from users.utils import LoginBlockUtil, MFABlockUtils
reason_password_failed = 'password_failed'
reason_password_decrypt_failed = 'password_decrypt_failed'
@@ -18,6 +16,7 @@ reason_user_not_exist = 'user_not_exist'
reason_password_expired = 'password_expired'
reason_user_invalid = 'user_invalid'
reason_user_inactive = 'user_inactive'
reason_user_expired = 'user_expired'
reason_backend_not_match = 'backend_not_match'
reason_acl_not_allow = 'acl_not_allow'
@@ -30,8 +29,9 @@ reason_choices = {
reason_password_expired: _("Password expired"),
reason_user_invalid: _('Disabled or expired'),
reason_user_inactive: _("This account is inactive."),
reason_user_expired: _("This account is expired"),
reason_backend_not_match: _("Auth backend not match"),
reason_acl_not_allow: _("ACL is not allowed")
reason_acl_not_allow: _("ACL is not allowed"),
}
old_reason_choices = {
'0': '-',
@@ -52,7 +52,15 @@ block_login_msg = _(
"The account has been locked "
"(please contact admin to unlock it or try again after {} minutes)"
)
mfa_failed_msg = _("MFA code invalid, or ntp sync server time")
block_mfa_msg = _(
"The account has been locked "
"(please contact admin to unlock it or try again after {} minutes)"
)
mfa_failed_msg = _(
"MFA code invalid, or ntp sync server time, "
"You can also try {times_try} times "
"(The account will be temporarily locked for {block_time} minutes)"
)
mfa_required_msg = _("MFA required")
mfa_unset_msg = _("MFA not set, please set it first")
@@ -80,7 +88,7 @@ class AuthFailedNeedBlockMixin:
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
increase_login_failed_count(self.username, self.ip)
LoginBlockUtil(self.username, self.ip).incr_failed_count()
class AuthFailedError(Exception):
@@ -107,13 +115,12 @@ class AuthFailedError(Exception):
class CredentialError(AuthFailedNeedLogMixin, AuthFailedNeedBlockMixin, AuthFailedError):
def __init__(self, error, username, ip, request):
super().__init__(error=error, username=username, ip=ip, request=request)
times_up = settings.SECURITY_LOGIN_LIMIT_COUNT
times_failed = get_login_failed_count(username, ip)
times_try = int(times_up) - int(times_failed)
util = LoginBlockUtil(username, ip)
times_remainder = util.get_remainder_times()
block_time = settings.SECURITY_LOGIN_LIMIT_TIME
default_msg = invalid_login_msg.format(
times_try=times_try, block_time=block_time
times_try=times_remainder, block_time=block_time
)
if error == reason_password_failed:
self.msg = default_msg
@@ -123,12 +130,32 @@ class CredentialError(AuthFailedNeedLogMixin, AuthFailedNeedBlockMixin, AuthFail
class MFAFailedError(AuthFailedNeedLogMixin, AuthFailedError):
error = reason_mfa_failed
msg = mfa_failed_msg
msg: str
def __init__(self, username, request):
def __init__(self, username, request, ip):
util = MFABlockUtils(username, ip)
util.incr_failed_count()
times_remainder = util.get_remainder_times()
block_time = settings.SECURITY_LOGIN_LIMIT_TIME
if times_remainder:
self.msg = mfa_failed_msg.format(
times_try=times_remainder, block_time=block_time
)
else:
self.msg = block_mfa_msg.format(settings.SECURITY_LOGIN_LIMIT_TIME)
super().__init__(username=username, request=request)
class BlockMFAError(AuthFailedNeedLogMixin, AuthFailedError):
error = 'block_mfa'
def __init__(self, username, request, ip):
self.msg = block_mfa_msg.format(settings.SECURITY_LOGIN_LIMIT_TIME)
super().__init__(username=username, request=request, ip=ip)
class MFAUnsetError(AuthFailedNeedLogMixin, AuthFailedError):
error = reason_mfa_unset
msg = mfa_unset_msg
@@ -184,6 +211,28 @@ class MFARequiredError(NeedMoreInfoError):
}
class ACLError(AuthFailedNeedLogMixin, AuthFailedError):
msg = reason_acl_not_allow
error = 'acl_error'
def __init__(self, msg, **kwargs):
self.msg = msg
super().__init__(**kwargs)
def as_data(self):
return {
"error": reason_acl_not_allow,
"msg": self.msg
}
class LoginIPNotAllowed(ACLError):
def __init__(self, username, request, **kwargs):
self.username = username
self.request = request
super().__init__(_("IP is not allowed"), **kwargs)
class LoginConfirmBaseError(NeedMoreInfoError):
def __init__(self, ticket_id, **kwargs):
self.ticket_id = ticket_id

View File

@@ -15,9 +15,7 @@ from django.shortcuts import reverse
from common.utils import get_object_or_none, get_request_ip, get_logger, bulk_get
from users.models import User
from users.utils import (
is_block_login, clean_failed_count
)
from users.utils import LoginBlockUtil, MFABlockUtils
from . import errors
from .utils import rsa_decrypt
from .signals import post_auth_success, post_auth_failed
@@ -117,7 +115,7 @@ class AuthMixin:
else:
username = self.request.POST.get("username")
ip = self.get_request_ip()
if is_block_login(username, ip):
if LoginBlockUtil(username, ip).is_block():
logger.warn('Ip was blocked' + ': ' + username + ':' + ip)
exception = errors.BlockLoginError(username=username, ip=ip)
if raise_exception:
@@ -173,7 +171,7 @@ class AuthMixin:
if not user:
self.raise_credential_error(errors.reason_password_failed)
elif user.is_expired:
self.raise_credential_error(errors.reason_user_inactive)
self.raise_credential_error(errors.reason_user_expired)
elif not user.is_active:
self.raise_credential_error(errors.reason_user_inactive)
return user
@@ -183,7 +181,7 @@ class AuthMixin:
from acls.models import LoginACL
is_allowed = LoginACL.allow_user_to_login(user, ip)
if not is_allowed:
raise self.raise_credential_error(error=errors.reason_acl_not_allow)
raise errors.LoginIPNotAllowed(username=user.username, request=self.request)
def check_user_auth(self, decrypt_passwd=False):
self.check_is_block()
@@ -197,7 +195,7 @@ class AuthMixin:
self._check_password_require_reset_or_not(user)
self._check_passwd_is_too_simple(user, password)
clean_failed_count(username, ip)
LoginBlockUtil(username, ip).clean_failed_count()
request.session['auth_password'] = 1
request.session['user_id'] = str(user.id)
request.session['auto_login'] = auto_login
@@ -253,15 +251,34 @@ class AuthMixin:
raise errors.MFAUnsetError(user, self.request, url)
raise errors.MFARequiredError()
def mark_mfa_ok(self):
self.request.session['auth_mfa'] = 1
self.request.session['auth_mfa_time'] = time.time()
self.request.session['auth_mfa_type'] = 'otp'
def check_mfa_is_block(self, username, ip, raise_exception=True):
if MFABlockUtils(username, ip).is_block():
logger.warn('Ip was blocked' + ': ' + username + ':' + ip)
exception = errors.BlockMFAError(username=username, request=self.request, ip=ip)
if raise_exception:
raise exception
else:
return exception
def check_user_mfa(self, code):
user = self.get_user_from_session()
ip = self.get_request_ip()
self.check_mfa_is_block(user.username, ip)
ok = user.check_mfa(code)
if ok:
self.request.session['auth_mfa'] = 1
self.request.session['auth_mfa_time'] = time.time()
self.request.session['auth_mfa_type'] = 'otp'
self.mark_mfa_ok()
return
raise errors.MFAFailedError(username=user.username, request=self.request)
raise errors.MFAFailedError(
username=user.username,
request=self.request,
ip=ip
)
def get_ticket(self):
from tickets.models import Ticket

View File

@@ -1,5 +1,6 @@
# -*- coding: utf-8 -*-
#
from django.utils import timezone
from rest_framework import serializers
from common.utils import get_object_or_none
@@ -44,6 +45,10 @@ class BearerTokenSerializer(serializers.Serializer):
def get_keyword(obj):
return 'Bearer'
def update_last_login(self, user):
user.last_login = timezone.now()
user.save(update_fields=['last_login'])
def create(self, validated_data):
request = self.context.get('request')
if request.user and not request.user.is_anonymous:
@@ -56,6 +61,8 @@ class BearerTokenSerializer(serializers.Serializer):
"user id {} not exist".format(user_id)
)
token, date_expired = user.create_bearer_token(request)
self.update_last_login(user)
instance = {
"token": token,
"date_expired": date_expired,

View File

@@ -23,4 +23,3 @@ urlpatterns = [
]
urlpatterns += router.urls

View File

@@ -55,6 +55,9 @@ class UserLoginView(mixins.AuthMixin, FormView):
def form_valid(self, form):
if not self.request.session.test_cookie_worked():
return HttpResponse(_("Please enable cookies and try again."))
# https://docs.djangoproject.com/en/3.1/topics/http/sessions/#setting-test-cookies
self.request.session.delete_test_cookie()
try:
self.check_user_auth(decrypt_passwd=True)
except errors.AuthFailedError as e:
@@ -66,6 +69,7 @@ class UserLoginView(mixins.AuthMixin, FormView):
new_form = form_cls(data=form.data)
new_form._errors = form.errors
context = self.get_context_data(form=new_form)
self.request.session.set_test_cookie()
return self.render_to_response(context)
except (errors.PasswdTooSimple, errors.PasswordRequireResetError) as e:
return redirect(e.url)

View File

@@ -22,10 +22,12 @@ class UserLoginOtpView(mixins.AuthMixin, FormView):
try:
self.check_user_mfa(otp_code)
return redirect_to_guard_view()
except errors.MFAFailedError as e:
except (errors.MFAFailedError, errors.BlockMFAError) as e:
form.add_error('otp_code', e.msg)
return super().form_invalid(form)
except Exception as e:
logger.error(e)
import traceback
traceback.print_exception()
return redirect_to_guard_view()

View File

@@ -143,5 +143,5 @@ class BaseFileParser(BaseParser):
return data
except Exception as e:
logger.error(e, exc_info=True)
raise ParseError('Parse error! ({})'.format(self.media_type))
raise ParseError(_('Parse file error: {}').format(e))

View File

@@ -0,0 +1,19 @@
from django.core.management.base import BaseCommand
from assets.signals_handler.node_assets_mapping import expire_node_assets_mapping_for_memory
from orgs.models import Organization
def expire_node_assets_mapping():
org_ids = Organization.objects.all().values_list('id', flat=True)
org_ids = [*org_ids, '00000000-0000-0000-0000-000000000000']
for org_id in org_ids:
expire_node_assets_mapping_for_memory(org_id)
class Command(BaseCommand):
help = 'Expire caches'
def handle(self, *args, **options):
expire_node_assets_mapping()

View File

@@ -9,6 +9,7 @@ from itertools import chain
from django.db.models.signals import m2m_changed
from django.core.cache import cache
from django.http import JsonResponse
from django.utils.translation import ugettext as _
from rest_framework.response import Response
from rest_framework.settings import api_settings
from rest_framework.decorators import action
@@ -47,6 +48,9 @@ class RenderToJsonMixin:
column_title_field_pairs = jms_context.get('column_title_field_pairs', ())
data['title'] = column_title_field_pairs
if isinstance(request.data, (list, tuple)) and not any(request.data):
error = _("Request file format may be wrong")
return Response(data={"error": error}, status=400)
return Response(data=data)

View File

@@ -1,12 +1,14 @@
# -*- coding: utf-8 -*-
#
# coding: utf-8
from django.contrib.auth.mixins import UserPassesTestMixin
from django.utils import timezone
__all__ = ["DatetimeSearchMixin"]
from rest_framework import permissions
class DatetimeSearchMixin:
date_format = '%Y-%m-%d'
@@ -36,3 +38,17 @@ class DatetimeSearchMixin:
def get(self, request, *args, **kwargs):
self.get_date_range()
return super().get(request, *args, **kwargs)
class PermissionsMixin(UserPassesTestMixin):
permission_classes = [permissions.IsAuthenticated]
def get_permissions(self):
return self.permission_classes
def test_func(self):
permission_classes = self.get_permissions()
for permission_class in permission_classes:
if not permission_class().has_permission(self.request, self):
return False
return True

View File

@@ -2,7 +2,6 @@
#
import time
from rest_framework import permissions
from django.contrib.auth.mixins import UserPassesTestMixin
from django.conf import settings
from orgs.utils import current_org
@@ -95,20 +94,6 @@ class WithBootstrapToken(permissions.BasePermission):
return settings.BOOTSTRAP_TOKEN == request_bootstrap_token
class PermissionsMixin(UserPassesTestMixin):
permission_classes = [permissions.IsAuthenticated]
def get_permissions(self):
return self.permission_classes
def test_func(self):
permission_classes = self.get_permissions()
for permission_class in permission_classes:
if not permission_class().has_permission(self.request, self):
return False
return True
class UserCanAnyPermCurrentOrg(permissions.BasePermission):
def has_permission(self, request, view):
return current_org.can_any_by(request.user)

View File

@@ -1 +1,2 @@

View File

@@ -1,3 +1,5 @@
import time
from django.core.cache import cache
from django.utils import timezone
from django.utils.timesince import timesince
@@ -6,6 +8,8 @@ from django.http.response import JsonResponse, HttpResponse
from rest_framework.views import APIView
from rest_framework.permissions import AllowAny
from collections import Counter
from django.conf import settings
from rest_framework.response import Response
from users.models import User
from assets.models import Asset
@@ -307,7 +311,68 @@ class IndexApi(TotalCountMixin, DatesLoginMetricMixin, APIView):
return JsonResponse(data, status=200)
class PrometheusMetricsApi(APIView):
class HealthApiMixin(APIView):
def is_token_right(self):
token = self.request.query_params.get('token')
ok_token = settings.HEALTH_CHECK_TOKEN
if ok_token and token != ok_token:
return False
return True
def check_permissions(self, request):
if not self.is_token_right():
msg = 'Health check token error, ' \
'Please set query param in url and same with setting HEALTH_CHECK_TOKEN. ' \
'eg: $PATH/?token=$HEALTH_CHECK_TOKEN'
self.permission_denied(request, message={'error': msg}, code=403)
class HealthCheckView(HealthApiMixin):
permission_classes = (AllowAny,)
@staticmethod
def get_db_status():
t1 = time.time()
try:
User.objects.first()
t2 = time.time()
return True, t2 - t1
except:
t2 = time.time()
return False, t2 - t1
def get_redis_status(self):
key = 'HEALTH_CHECK'
t1 = time.time()
try:
value = '1'
cache.set(key, '1', 10)
got = cache.get(key)
t2 = time.time()
if value == got:
return True, t2 -t1
return False, t2 -t1
except:
t2 = time.time()
return False, t2 - t1
def get(self, request):
redis_status, redis_time = self.get_redis_status()
db_status, db_time = self.get_db_status()
status = all([redis_status, db_status])
data = {
'status': status,
'db_status': db_status,
'db_time': db_time,
'redis_status': redis_status,
'redis_time': redis_time,
'time': int(time.time())
}
return Response(data)
class PrometheusMetricsApi(HealthApiMixin):
permission_classes = (AllowAny,)
def get(self, request, *args, **kwargs):

View File

@@ -143,6 +143,7 @@ class Config(dict):
'REDIS_DB_SESSION': 5,
'REDIS_DB_WS': 6,
'GLOBAL_ORG_DISPLAY_NAME': '',
'SITE_URL': 'http://localhost:8080',
'CAPTCHA_TEST_MODE': None,
'TOKEN_EXPIRATION': 3600 * 24,
@@ -267,7 +268,7 @@ class Config(dict):
'WINDOWS_SSH_DEFAULT_SHELL': 'cmd',
'FLOWER_URL': "127.0.0.1:5555",
'DEFAULT_ORG_SHOW_ALL_USERS': True,
'PERIOD_TASK_ENABLE': True,
'PERIOD_TASK_ENABLED': True,
'FORCE_SCRIPT_NAME': '',
'LOGIN_CONFIRM_ENABLE': False,
'WINDOWS_SKIP_ALL_MANUAL_PASSWORD': False,
@@ -288,6 +289,7 @@ class Config(dict):
'SESSION_SAVE_EVERY_REQUEST': True,
'SESSION_EXPIRE_AT_BROWSER_CLOSE_FORCE': False,
'FORGOT_PASSWORD_URL': '',
'HEALTH_CHECK_TOKEN': ''
}
def compatible_auth_openid_of_key(self):

View File

@@ -120,3 +120,7 @@ CONNECTION_TOKEN_ENABLED = CONFIG.CONNECTION_TOKEN_ENABLED
DISK_CHECK_ENABLED = CONFIG.DISK_CHECK_ENABLED
FORGOT_PASSWORD_URL = CONFIG.FORGOT_PASSWORD_URL
# 自定义默认组织名
GLOBAL_ORG_DISPLAY_NAME = CONFIG.GLOBAL_ORG_DISPLAY_NAME
HEALTH_CHECK_TOKEN = CONFIG.HEALTH_CHECK_TOKEN

View File

@@ -48,7 +48,8 @@ urlpatterns = [
path('', views.IndexView.as_view(), name='index'),
path('api/v1/', include(api_v1)),
re_path('api/(?P<app>\w+)/(?P<version>v\d)/.*', views.redirect_format_api),
path('api/health/', views.HealthCheckView.as_view(), name="health"),
path('api/health/', api.HealthCheckView.as_view(), name="health"),
path('api/v1/health/', api.HealthCheckView.as_view(), name="health_v1"),
# External apps url
path('core/auth/captcha/', include('captcha.urls')),
path('core/', include(app_view_patterns)),

View File

@@ -1,6 +1,7 @@
from django.views.generic import TemplateView
from django.shortcuts import redirect
from common.permissions import PermissionsMixin, IsValidUser
from common.permissions import IsValidUser
from common.mixins.views import PermissionsMixin
__all__ = ['IndexView']

View File

@@ -17,7 +17,7 @@ from common.http import HttpResponseTemporaryRedirect
__all__ = [
'LunaView', 'I18NView', 'KokoView', 'WsView', 'HealthCheckView',
'LunaView', 'I18NView', 'KokoView', 'WsView',
'redirect_format_api', 'redirect_old_apps_view', 'UIView'
]
@@ -64,13 +64,6 @@ def redirect_old_apps_view(request, *args, **kwargs):
return HttpResponseTemporaryRedirect(new_path)
class HealthCheckView(APIView):
permission_classes = (AllowAny,)
def get(self, request):
return JsonResponse({"status": 1, "time": int(time.time())})
class WsView(APIView):
ws_port = settings.HTTP_LISTEN_PORT + 1

Binary file not shown.

File diff suppressed because it is too large Load Diff

View File

@@ -72,6 +72,7 @@ def create_or_update_celery_periodic_tasks(tasks):
crontab=crontab,
name=name,
task=detail['task'],
enabled=detail.get('enabled', True),
args=json.dumps(detail.get('args', [])),
kwargs=json.dumps(detail.get('kwargs', {})),
description=detail.get('description') or ''

View File

@@ -3,8 +3,8 @@
from django.views.generic import TemplateView
from django.conf import settings
from common.permissions import PermissionsMixin, IsOrgAdmin, IsOrgAuditor
from common.permissions import IsOrgAdmin, IsOrgAuditor
from common.mixins.views import PermissionsMixin
__all__ = ['CeleryTaskLogView']

View File

@@ -90,6 +90,11 @@ class OrgMemberRelationBulkViewSet(JMSBulkRelationModelViewSet):
filterset_class = OrgMemberRelationFilterSet
search_fields = ('user__name', 'user__username', 'org__name')
def get_queryset(self):
queryset = super().get_queryset()
queryset = queryset.exclude(user__role=User.ROLE.APP)
return queryset
def perform_bulk_destroy(self, queryset):
objs = list(queryset.all().prefetch_related('user', 'org'))
queryset.delete()

View File

@@ -7,7 +7,7 @@ from django.db.models import signals
from django.db.models import Q
from django.utils.translation import ugettext_lazy as _
from common.utils import is_uuid, lazyproperty
from common.utils import lazyproperty, settings
from common.const import choices
from common.db.models import ChoiceSet
@@ -193,7 +193,8 @@ class Organization(models.Model):
@classmethod
def root(cls):
return cls(id=cls.ROOT_ID, name=cls.ROOT_NAME)
name = settings.GLOBAL_ORG_DISPLAY_NAME or cls.ROOT_NAME
return cls(id=cls.ROOT_ID, name=name)
def is_root(self):
return self.id == self.ROOT_ID

View File

@@ -46,12 +46,19 @@ def subscribe_orgs_mapping_expire(sender, **kwargs):
logger.debug("Start subscribe for expire orgs mapping from memory")
def keep_subscribe():
subscribe = orgs_mapping_for_memory_pub_sub.subscribe()
for message in subscribe.listen():
if message['type'] != 'message':
continue
Organization.expire_orgs_mapping()
logger.debug('Expire orgs mapping')
while True:
try:
subscribe = orgs_mapping_for_memory_pub_sub.subscribe()
for message in subscribe.listen():
if message['type'] != 'message':
continue
if message['data'] == b'error':
raise ValueError
Organization.expire_orgs_mapping()
logger.debug('Expire orgs mapping')
except Exception as e:
logger.exception(f'subscribe_orgs_mapping_expire: {e}')
Organization.expire_orgs_mapping()
t = threading.Thread(target=keep_subscribe)
t.daemon = True

View File

@@ -20,3 +20,4 @@ class AssetPermissionViewSet(OrgBulkModelViewSet):
model = AssetPermission
serializer_class = serializers.AssetPermissionSerializer
filterset_class = AssetPermissionFilter
search_fields = ('name',)

View File

@@ -4,7 +4,7 @@
from rest_framework import serializers
from django.utils.translation import ugettext_lazy as _
from assets.models import Node, SystemUser, Asset
from assets.models import Node, SystemUser, Asset, Platform
from assets.serializers import ProtocolsField
from perms.serializers.asset.permission import ActionsField
@@ -39,7 +39,9 @@ class AssetGrantedSerializer(serializers.ModelSerializer):
被授权资产的数据结构
"""
protocols = ProtocolsField(label=_('Protocols'), required=False, read_only=True)
platform = serializers.ReadOnlyField(source='platform_base')
platform = serializers.SlugRelatedField(
slug_field='name', queryset=Platform.objects.all(), label=_("Platform")
)
class Meta:
model = Asset

View File

@@ -488,11 +488,12 @@ class UserGrantedAssetsQueryUtils(UserGrantedUtilsBase):
if granted_status == NodeFrom.granted:
assets = Asset.objects.order_by().filter(nodes__id=node.id)
return assets
elif granted_status == NodeFrom.asset:
return self._get_indirect_granted_node_assets(node.id)
assets = self._get_indirect_granted_node_assets(node.id)
else:
return Asset.objects.none()
assets = Asset.objects.none()
assets = assets.order_by('hostname')
return assets
def _get_indirect_granted_node_assets(self, id) -> AssetQuerySet:
assets = Asset.objects.order_by().filter(nodes__id=id).distinct() & self.get_direct_granted_assets()
@@ -538,6 +539,10 @@ class UserGrantedAssetsQueryUtils(UserGrantedUtilsBase):
class UserGrantedNodesQueryUtils(UserGrantedUtilsBase):
def sort(self, nodes):
nodes = sorted(nodes, key=lambda x: x.value)
return nodes
def get_node_children(self, key):
if not key:
return self.get_top_level_nodes()
@@ -545,11 +550,13 @@ class UserGrantedNodesQueryUtils(UserGrantedUtilsBase):
node = PermNode.objects.get(key=key)
granted_status = node.get_granted_status(self.user)
if granted_status == NodeFrom.granted:
return PermNode.objects.filter(parent_key=key)
nodes = PermNode.objects.filter(parent_key=key)
elif granted_status in (NodeFrom.asset, NodeFrom.child):
return self.get_indirect_granted_node_children(key)
nodes = self.get_indirect_granted_node_children(key)
else:
return PermNode.objects.none()
nodes = PermNode.objects.none()
nodes = self.sort(nodes)
return nodes
def get_indirect_granted_node_children(self, key):
"""
@@ -571,7 +578,8 @@ class UserGrantedNodesQueryUtils(UserGrantedUtilsBase):
def get_top_level_nodes(self):
nodes = self.get_special_nodes()
nodes.extend(self.get_indirect_granted_node_children(''))
real_nodes = self.get_indirect_granted_node_children('')
nodes.extend(self.sort(real_nodes))
return nodes
def get_ungrouped_node(self):

View File

@@ -24,6 +24,10 @@ class BasicSettingSerializer(serializers.Serializer):
help_text=_('The forgot password url on login page, If you use '
'ldap or cas external authentication, you can set it')
)
GLOBAL_ORG_DISPLAY_NAME = serializers.CharField(
required=False, max_length=1024, allow_blank=True, allow_null=True, label=_("Global organization name"),
help_text=_('The name of global organization to display')
)
class EmailSettingSerializer(serializers.Serializer):

View File

@@ -6,6 +6,7 @@ import threading
from django.dispatch import receiver
from django.db.models.signals import post_save, pre_save
from django.utils.functional import LazyObject
from django.db import close_old_connections
from jumpserver.utils import current_request
from common.decorator import on_transaction_commit
@@ -71,13 +72,21 @@ def subscribe_settings_change(sender, **kwargs):
logger.debug("Start subscribe setting change")
def keep_subscribe():
sub = setting_pub_sub.subscribe()
for msg in sub.listen():
if msg["type"] != "message":
continue
item = msg['data'].decode()
logger.debug("Found setting change: {}".format(str(item)))
Setting.refresh_item(item)
while True:
try:
sub = setting_pub_sub.subscribe()
for msg in sub.listen():
close_old_connections()
if msg["type"] != "message":
continue
item = msg['data'].decode()
logger.debug("Found setting change: {}".format(str(item)))
Setting.refresh_item(item)
except Exception as e:
logger.exception(f'subscribe_settings_change: {e}')
close_old_connections()
Setting.refresh_all_settings()
t = threading.Thread(target=keep_subscribe)
t.daemon = True
t.start()

View File

@@ -5,4 +5,4 @@ from .session import *
from .command import *
from .task import *
from .storage import *
from .component import *
from .status import *

View File

@@ -11,6 +11,7 @@ from rest_framework.response import Response
from rest_framework.decorators import action
from django.template import loader
from common.http import is_true
from terminal.models import CommandStorage, Command
from terminal.filters import CommandFilter
from orgs.utils import current_org
@@ -140,7 +141,19 @@ class CommandViewSet(viewsets.ModelViewSet):
if session_id and not command_storage_id:
# 会话里的命令列表肯定会提供 session_id这里防止 merge 的时候取全量的数据
return self.merge_all_storage_list(request, *args, **kwargs)
return super().list(request, *args, **kwargs)
queryset = self.filter_queryset(self.get_queryset())
page = self.paginate_queryset(queryset)
if page is not None:
serializer = self.get_serializer(page, many=True)
return self.get_paginated_response(serializer.data)
# 适配像 ES 这种没有指定分页只返回少量数据的情况
queryset = queryset[:]
serializer = self.get_serializer(queryset, many=True)
return Response(serializer.data)
def get_queryset(self):
command_storage_id = self.request.query_params.get('command_storage_id')

View File

@@ -1,34 +0,0 @@
# -*- coding: utf-8 -*-
#
import logging
from rest_framework import generics, status
from rest_framework.views import Response
from .. import serializers
from ..utils import ComponentsMetricsUtil
from common.permissions import IsAppUser, IsSuperUser
logger = logging.getLogger(__file__)
__all__ = [
'ComponentsStateAPIView', 'ComponentsMetricsAPIView',
]
class ComponentsStateAPIView(generics.CreateAPIView):
""" koko, guacamole, omnidb 上报状态 """
permission_classes = (IsAppUser,)
serializer_class = serializers.ComponentsStateSerializer
class ComponentsMetricsAPIView(generics.GenericAPIView):
""" 返回汇总组件指标数据 """
permission_classes = (IsSuperUser,)
def get(self, request, *args, **kwargs):
tp = request.query_params.get('type')
util = ComponentsMetricsUtil()
metrics = util.get_metrics(tp)
return Response(metrics, status=status.HTTP_200_OK)

View File

@@ -0,0 +1,74 @@
# -*- coding: utf-8 -*-
#
import logging
from django.shortcuts import get_object_or_404
from rest_framework import viewsets, generics
from rest_framework.views import Response
from rest_framework import status
from common.permissions import IsAppUser, IsOrgAdminOrAppUser, IsSuperUser
from ..models import Terminal, Status, Session
from .. import serializers
from ..utils import TypedComponentsStatusMetricsUtil
logger = logging.getLogger(__file__)
__all__ = [
'StatusViewSet',
'ComponentsMetricsAPIView',
]
class StatusViewSet(viewsets.ModelViewSet):
queryset = Status.objects.all()
serializer_class = serializers.StatusSerializer
permission_classes = (IsOrgAdminOrAppUser,)
session_serializer_class = serializers.SessionSerializer
task_serializer_class = serializers.TaskSerializer
def create(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
self.handle_sessions()
self.perform_create(serializer)
tasks = self.request.user.terminal.task_set.filter(is_finished=False)
serializer = self.task_serializer_class(tasks, many=True)
return Response(serializer.data, status=201)
def handle_sessions(self):
session_ids = self.request.data.get('sessions', [])
# guacamole 上报的 session 是字符串
# "[53cd3e47-210f-41d8-b3c6-a184f3, 53cd3e47-210f-41d8-b3c6-a184f4]"
if isinstance(session_ids, str):
session_ids = session_ids[1:-1].split(',')
session_ids = [sid.strip() for sid in session_ids if sid.strip()]
Session.set_sessions_active(session_ids)
def get_queryset(self):
terminal_id = self.kwargs.get("terminal", None)
if terminal_id:
terminal = get_object_or_404(Terminal, id=terminal_id)
return terminal.status_set.all()
return super().get_queryset()
def perform_create(self, serializer):
serializer.validated_data.pop('sessions', None)
serializer.validated_data["terminal"] = self.request.user.terminal
return super().perform_create(serializer)
def get_permissions(self):
if self.action == "create":
self.permission_classes = (IsAppUser,)
return super().get_permissions()
class ComponentsMetricsAPIView(generics.GenericAPIView):
""" 返回汇总组件指标数据 """
permission_classes = (IsSuperUser,)
def get(self, request, *args, **kwargs):
util = TypedComponentsStatusMetricsUtil()
metrics = util.get_metrics()
return Response(metrics, status=status.HTTP_200_OK)

View File

@@ -4,8 +4,7 @@ import logging
import uuid
from django.core.cache import cache
from django.shortcuts import get_object_or_404
from rest_framework import viewsets, generics
from rest_framework import generics
from rest_framework.views import APIView, Response
from rest_framework import status
from django.conf import settings
@@ -13,13 +12,13 @@ from django.conf import settings
from common.drf.api import JMSBulkModelViewSet
from common.utils import get_object_or_none
from common.permissions import IsAppUser, IsOrgAdminOrAppUser, IsSuperUser, WithBootstrapToken
from ..models import Terminal, Status, Session
from common.permissions import IsAppUser, IsSuperUser, WithBootstrapToken
from ..models import Terminal
from .. import serializers
from .. import exceptions
__all__ = [
'TerminalViewSet', 'StatusViewSet', 'TerminalConfig',
'TerminalViewSet', 'TerminalConfig',
'TerminalRegistrationApi',
]
logger = logging.getLogger(__file__)
@@ -72,45 +71,6 @@ class TerminalViewSet(JMSBulkModelViewSet):
return queryset
class StatusViewSet(viewsets.ModelViewSet):
queryset = Status.objects.all()
serializer_class = serializers.StatusSerializer
permission_classes = (IsOrgAdminOrAppUser,)
session_serializer_class = serializers.SessionSerializer
task_serializer_class = serializers.TaskSerializer
def create(self, request, *args, **kwargs):
self.handle_sessions()
tasks = self.request.user.terminal.task_set.filter(is_finished=False)
serializer = self.task_serializer_class(tasks, many=True)
return Response(serializer.data, status=201)
def handle_sessions(self):
session_ids = self.request.data.get('sessions', [])
# guacamole 上报的 session 是字符串
# "[53cd3e47-210f-41d8-b3c6-a184f3, 53cd3e47-210f-41d8-b3c6-a184f4]"
if isinstance(session_ids, str):
session_ids = session_ids[1:-1].split(',')
session_ids = [sid.strip() for sid in session_ids if sid.strip()]
Session.set_sessions_active(session_ids)
def get_queryset(self):
terminal_id = self.kwargs.get("terminal", None)
if terminal_id:
terminal = get_object_or_404(Terminal, id=terminal_id)
self.queryset = terminal.status_set.all()
return self.queryset
def perform_create(self, serializer):
serializer.validated_data["terminal"] = self.request.user.terminal
return super().perform_create(serializer)
def get_permissions(self):
if self.action == "create":
self.permission_classes = (IsAppUser,)
return super().get_permissions()
class TerminalConfig(APIView):
permission_classes = (IsAppUser,)

View File

@@ -10,6 +10,7 @@ import inspect
from django.db.models import QuerySet as DJQuerySet
from elasticsearch import Elasticsearch
from elasticsearch.helpers import bulk
from elasticsearch.exceptions import RequestError
from common.utils.common import lazyproperty
from common.utils import get_logger
@@ -25,8 +26,21 @@ class CommandStore():
kwargs = config.get("OTHER", {})
self.index = config.get("INDEX") or 'jumpserver'
self.doc_type = config.get("DOC_TYPE") or 'command_store'
ignore_verify_certs = kwargs.pop('ignore_verify_certs', False)
if ignore_verify_certs:
kwargs['verify_certs'] = None
self.es = Elasticsearch(hosts=hosts, max_retries=0, **kwargs)
def pre_use_check(self):
self._ensure_index_exists()
def _ensure_index_exists(self):
try:
self.es.indices.create(self.index)
except RequestError:
pass
@staticmethod
def make_data(command):
data = dict(
@@ -230,6 +244,7 @@ class QuerySet(DJQuerySet):
uqs = QuerySet(self._command_store_config)
uqs._method_calls = self._method_calls.copy()
uqs._slice = self._slice
uqs.model = self.model
return uqs
def count(self, limit_to_max_result_window=True):

View File

@@ -31,6 +31,7 @@ class ComponentStatusChoices(TextChoices):
critical = 'critical', _('Critical')
high = 'high', _('High')
normal = 'normal', _('Normal')
offline = 'offline', _('Offline')
@classmethod
def status(cls):

View File

@@ -0,0 +1,18 @@
# Generated by Django 3.1 on 2021-03-24 02:08
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('terminal', '0032_auto_20210302_1853'),
]
operations = [
migrations.AlterField(
model_name='session',
name='login_from',
field=models.CharField(choices=[('ST', 'SSH Terminal'), ('RT', 'RDP Terminal'), ('WT', 'Web Terminal')], default='ST', max_length=2, verbose_name='Login from'),
),
]

View File

@@ -0,0 +1,42 @@
# Generated by Django 3.1 on 2021-04-06 06:34
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('terminal', '0033_auto_20210324_1008'),
]
operations = [
migrations.RemoveField(
model_name='status',
name='cpu_used',
),
migrations.AddField(
model_name='status',
name='cpu_load',
field=models.FloatField(default=0, verbose_name='CPU Load'),
),
migrations.AddField(
model_name='status',
name='disk_used',
field=models.FloatField(default=0, verbose_name='Disk Used'),
),
migrations.AlterField(
model_name='status',
name='boot_time',
field=models.FloatField(default=0, verbose_name='Boot Time'),
),
migrations.AlterField(
model_name='status',
name='connections',
field=models.IntegerField(default=0, verbose_name='Connections'),
),
migrations.AlterField(
model_name='status',
name='threads',
field=models.IntegerField(default=0, verbose_name='Threads'),
),
]

View File

@@ -14,12 +14,12 @@ from assets.models import Asset
from orgs.mixins.models import OrgModelMixin
from common.db.models import ChoiceSet
from ..backends import get_multi_command_storage
from .terminal import Terminal
class Session(OrgModelMixin):
class LOGIN_FROM(ChoiceSet):
ST = 'ST', 'SSH Terminal'
RT = 'RT', 'RDP Terminal'
WT = 'WT', 'Web Terminal'
class PROTOCOL(ChoiceSet):
@@ -46,7 +46,7 @@ class Session(OrgModelMixin):
is_finished = models.BooleanField(default=False, db_index=True)
has_replay = models.BooleanField(default=False, verbose_name=_("Replay"))
has_command = models.BooleanField(default=False, verbose_name=_("Command"))
terminal = models.ForeignKey(Terminal, null=True, on_delete=models.DO_NOTHING, db_constraint=False)
terminal = models.ForeignKey('terminal.Terminal', null=True, on_delete=models.DO_NOTHING, db_constraint=False)
protocol = models.CharField(choices=PROTOCOL.choices, default='ssh', max_length=16, db_index=True)
date_start = models.DateTimeField(verbose_name=_("Date start"), db_index=True, default=timezone.now)
date_end = models.DateTimeField(verbose_name=_("Date end"), null=True)

View File

@@ -3,26 +3,62 @@ from __future__ import unicode_literals
import uuid
from django.db import models
from django.forms.models import model_to_dict
from django.core.cache import cache
from django.utils.translation import ugettext_lazy as _
from .terminal import Terminal
from common.utils import get_logger
logger = get_logger(__name__)
class Status(models.Model):
id = models.UUIDField(default=uuid.uuid4, primary_key=True)
session_online = models.IntegerField(verbose_name=_("Session Online"), default=0)
cpu_used = models.FloatField(verbose_name=_("CPU Usage"))
cpu_load = models.FloatField(verbose_name=_("CPU Load"), default=0)
memory_used = models.FloatField(verbose_name=_("Memory Used"))
connections = models.IntegerField(verbose_name=_("Connections"))
threads = models.IntegerField(verbose_name=_("Threads"))
boot_time = models.FloatField(verbose_name=_("Boot Time"))
terminal = models.ForeignKey(Terminal, null=True, on_delete=models.CASCADE)
disk_used = models.FloatField(verbose_name=_("Disk Used"), default=0)
connections = models.IntegerField(verbose_name=_("Connections"), default=0)
threads = models.IntegerField(verbose_name=_("Threads"), default=0)
boot_time = models.FloatField(verbose_name=_("Boot Time"), default=0)
terminal = models.ForeignKey('terminal.Terminal', null=True, on_delete=models.CASCADE)
date_created = models.DateTimeField(auto_now_add=True)
CACHE_KEY = 'TERMINAL_STATUS_{}'
class Meta:
db_table = 'terminal_status'
get_latest_by = 'date_created'
def __str__(self):
return self.date_created.strftime("%Y-%m-%d %H:%M:%S")
def save_to_cache(self):
if not self.terminal:
return
key = self.CACHE_KEY.format(self.terminal.id)
data = model_to_dict(self)
cache.set(key, data, 60*3)
return data
@classmethod
def get_terminal_latest_status(cls, terminal):
from ..utils import ComputeStatUtil
stat = cls.get_terminal_latest_stat(terminal)
return ComputeStatUtil.compute_component_status(stat)
@classmethod
def get_terminal_latest_stat(cls, terminal):
key = cls.CACHE_KEY.format(terminal.id)
data = cache.get(key)
if not data:
return None
data.pop('terminal', None)
stat = cls(**data)
stat.terminal = terminal
return stat
def save(self, force_insert=False, force_update=False, using=None,
update_fields=None):
self.terminal.set_alive(ttl=120)
return self.save_to_cache()
# return super().save()

View File

@@ -76,6 +76,15 @@ class CommandStorage(CommonModelMixin):
qs.model = Command
return qs
def save(self, force_insert=False, force_update=False, using=None,
update_fields=None):
super().save()
if self.type in TYPE_ENGINE_MAPPING:
engine_mod = import_module(TYPE_ENGINE_MAPPING[self.type])
backend = engine_mod.CommandStore(self.config)
backend.pre_use_check()
class ReplayStorage(CommonModelMixin):
name = models.CharField(max_length=128, verbose_name=_("Name"), unique=True)

View File

@@ -1,168 +1,64 @@
from __future__ import unicode_literals
import uuid
from django.db import models
from django.core.cache import cache
from django.utils.translation import ugettext_lazy as _
from django.conf import settings
from django.core.cache import cache
from common.utils import get_logger
from users.models import User
from orgs.utils import tmp_to_root_org
from .status import Status
from .. import const
from ..const import ComponentStatusChoices as StatusChoice
from .session import Session
logger = get_logger(__file__)
class ComputeStatusMixin:
# system status
@staticmethod
def _common_compute_system_status(value, thresholds):
if thresholds[0] <= value <= thresholds[1]:
return const.ComponentStatusChoices.normal.value
elif thresholds[1] < value <= thresholds[2]:
return const.ComponentStatusChoices.high.value
else:
return const.ComponentStatusChoices.critical.value
def _compute_system_cpu_load_1_status(self, value):
thresholds = [0, 5, 20]
return self._common_compute_system_status(value, thresholds)
def _compute_system_memory_used_percent_status(self, value):
thresholds = [0, 85, 95]
return self._common_compute_system_status(value, thresholds)
def _compute_system_disk_used_percent_status(self, value):
thresholds = [0, 80, 99]
return self._common_compute_system_status(value, thresholds)
def _compute_system_status(self, state):
system_status_keys = [
'system_cpu_load_1', 'system_memory_used_percent', 'system_disk_used_percent'
]
system_status = []
for system_status_key in system_status_keys:
state_value = state.get(system_status_key)
if state_value is None:
msg = 'state: {}, state_key: {}, state_value: {}'
logger.debug(msg.format(state, system_status_key, state_value))
state_value = 0
status = getattr(self, f'_compute_{system_status_key}_status')(state_value)
system_status.append(status)
return system_status
def _compute_component_status(self, state):
system_status = self._compute_system_status(state)
if const.ComponentStatusChoices.critical in system_status:
return const.ComponentStatusChoices.critical
elif const.ComponentStatusChoices.high in system_status:
return const.ComponentStatusChoices.high
else:
return const.ComponentStatusChoices.normal
@staticmethod
def _compute_component_status_display(status):
return getattr(const.ComponentStatusChoices, status).label
class TerminalStateMixin(ComputeStatusMixin):
CACHE_KEY_COMPONENT_STATE = 'CACHE_KEY_COMPONENT_STATE_TERMINAL_{}'
CACHE_TIMEOUT = 120
class TerminalStatusMixin:
ALIVE_KEY = 'TERMINAL_ALIVE_{}'
id: str
@property
def cache_key(self):
return self.CACHE_KEY_COMPONENT_STATE.format(str(self.id))
# get
def _get_from_cache(self):
return cache.get(self.cache_key)
def _set_to_cache(self, state):
cache.set(self.cache_key, state, self.CACHE_TIMEOUT)
# set
def _add_status(self, state):
status = self._compute_component_status(state)
status_display = self._compute_component_status_display(status)
state.update({
'status': status,
'status_display': status_display
})
def latest_status(self):
return Status.get_terminal_latest_status(self)
@property
def state(self):
state = self._get_from_cache()
return state or {}
@state.setter
def state(self, state):
self._add_status(state)
self._set_to_cache(state)
class TerminalStatusMixin(TerminalStateMixin):
# alive
@property
def is_alive(self):
return bool(self.state)
# status
@property
def status(self):
if self.is_alive:
return self.state['status']
else:
return const.ComponentStatusChoices.critical.value
def latest_status_display(self):
return self.latest_status.label
@property
def status_display(self):
return self._compute_component_status_display(self.status)
def latest_stat(self):
return Status.get_terminal_latest_stat(self)
@property
def is_normal(self):
return self.status == const.ComponentStatusChoices.normal.value
return self.latest_status == StatusChoice.normal
@property
def is_high(self):
return self.status == const.ComponentStatusChoices.high.value
return self.latest_status == StatusChoice.high
@property
def is_critical(self):
return self.status == const.ComponentStatusChoices.critical.value
class Terminal(TerminalStatusMixin, models.Model):
id = models.UUIDField(default=uuid.uuid4, primary_key=True)
name = models.CharField(max_length=128, verbose_name=_('Name'))
type = models.CharField(
choices=const.TerminalTypeChoices.choices, default=const.TerminalTypeChoices.koko.value,
max_length=64, verbose_name=_('type')
)
remote_addr = models.CharField(max_length=128, blank=True, verbose_name=_('Remote Address'))
ssh_port = models.IntegerField(verbose_name=_('SSH Port'), default=2222)
http_port = models.IntegerField(verbose_name=_('HTTP Port'), default=5000)
command_storage = models.CharField(max_length=128, verbose_name=_("Command storage"), default='default')
replay_storage = models.CharField(max_length=128, verbose_name=_("Replay storage"), default='default')
user = models.OneToOneField(User, related_name='terminal', verbose_name='Application User', null=True, on_delete=models.CASCADE)
is_accepted = models.BooleanField(default=False, verbose_name='Is Accepted')
is_deleted = models.BooleanField(default=False)
date_created = models.DateTimeField(auto_now_add=True)
comment = models.TextField(blank=True, verbose_name=_('Comment'))
return self.latest_status == StatusChoice.critical
@property
def is_active(self):
if self.user and self.user.is_active:
return True
return False
def is_alive(self):
key = self.ALIVE_KEY.format(self.id)
# return self.latest_status != StatusChoice.offline
return cache.get(key, False)
@is_active.setter
def is_active(self, active):
if self.user:
self.user.is_active = active
self.user.save()
def set_alive(self, ttl=120):
key = self.ALIVE_KEY.format(self.id)
cache.set(key, True, ttl)
class StorageMixin:
command_storage: str
replay_storage: str
def get_command_storage(self):
from .storage import CommandStorage
@@ -198,6 +94,44 @@ class Terminal(TerminalStatusMixin, models.Model):
config = self.get_replay_storage_config()
return {"TERMINAL_REPLAY_STORAGE": config}
class Terminal(StorageMixin, TerminalStatusMixin, models.Model):
id = models.UUIDField(default=uuid.uuid4, primary_key=True)
name = models.CharField(max_length=128, verbose_name=_('Name'))
type = models.CharField(
choices=const.TerminalTypeChoices.choices, default=const.TerminalTypeChoices.koko.value,
max_length=64, verbose_name=_('type')
)
remote_addr = models.CharField(max_length=128, blank=True, verbose_name=_('Remote Address'))
ssh_port = models.IntegerField(verbose_name=_('SSH Port'), default=2222)
http_port = models.IntegerField(verbose_name=_('HTTP Port'), default=5000)
command_storage = models.CharField(max_length=128, verbose_name=_("Command storage"), default='default')
replay_storage = models.CharField(max_length=128, verbose_name=_("Replay storage"), default='default')
user = models.OneToOneField(User, related_name='terminal', verbose_name='Application User', null=True, on_delete=models.CASCADE)
is_accepted = models.BooleanField(default=False, verbose_name='Is Accepted')
is_deleted = models.BooleanField(default=False)
date_created = models.DateTimeField(auto_now_add=True)
comment = models.TextField(blank=True, verbose_name=_('Comment'))
@property
def is_active(self):
if self.user and self.user.is_active:
return True
return False
@is_active.setter
def is_active(self, active):
if self.user:
self.user.is_active = active
self.user.save()
def get_online_sessions(self):
with tmp_to_root_org():
return Session.objects.filter(terminal=self, is_finished=False)
def get_online_session_count(self):
return self.get_online_sessions().count()
@staticmethod
def get_login_title_setting():
login_title = None

View File

@@ -4,4 +4,3 @@ from .terminal import *
from .session import *
from .storage import *
from .command import *
from .components import *

View File

@@ -1,25 +0,0 @@
from rest_framework import serializers
from django.utils.translation import ugettext_lazy as _
class ComponentsStateSerializer(serializers.Serializer):
# system
system_cpu_load_1 = serializers.FloatField(
required=False, label=_("System cpu load (1 minutes)")
)
system_memory_used_percent = serializers.FloatField(
required=False, label=_('System memory used percent')
)
system_disk_used_percent = serializers.FloatField(
required=False, label=_('System disk used percent')
)
# sessions
session_active_count = serializers.IntegerField(
required=False, label=_("Session active count")
)
def save(self, **kwargs):
request = self.context['request']
terminal = request.user.terminal
terminal.state = self.validated_data

View File

@@ -181,7 +181,10 @@ class CommandStorageTypeESSerializer(serializers.Serializer):
max_length=1024, default='jumpserver', label=_('Index'), allow_null=True
)
DOC_TYPE = ReadableHiddenField(default='command', label=_('Doc type'), allow_null=True)
ignore_verify_certs = serializers.BooleanField(
default=False, label=_('Ignore Certificate Verification'),
source='OTHER.ignore_verify_certs', allow_null=True,
)
# mapping

View File

@@ -9,15 +9,34 @@ from common.utils import get_request_ip
from ..models import (
Terminal, Status, Session, Task, CommandStorage, ReplayStorage
)
from .components import ComponentsStateSerializer
class StatusSerializer(serializers.ModelSerializer):
sessions = serializers.ListSerializer(
child=serializers.CharField(max_length=36), write_only=True
)
class Meta:
fields = [
'id',
'cpu_load', 'memory_used', 'disk_used',
'session_online', 'sessions',
'terminal', 'date_created',
]
extra_kwargs = {
"cpu_load": {'default': 0},
"memory_used": {'default': 0},
"disk_used": {'default': 0},
}
model = Status
class TerminalSerializer(BulkModelSerializer):
session_online = serializers.SerializerMethodField()
session_online = serializers.ReadOnlyField(source='get_online_session_count')
is_alive = serializers.BooleanField(read_only=True)
status = serializers.CharField(read_only=True)
status_display = serializers.CharField(read_only=True)
state = ComponentsStateSerializer(read_only=True)
status = serializers.CharField(read_only=True, source='latest_status')
status_display = serializers.CharField(read_only=True, source='latest_status_display')
stat = StatusSerializer(read_only=True, source='latest_stat')
class Meta:
model = Terminal
@@ -25,7 +44,7 @@ class TerminalSerializer(BulkModelSerializer):
'id', 'name', 'type', 'remote_addr', 'http_port', 'ssh_port',
'comment', 'is_accepted', "is_active", 'session_online',
'is_alive', 'date_created', 'command_storage', 'replay_storage',
'status', 'status_display', 'state'
'status', 'status_display', 'stat'
]
read_only_fields = ['type', 'date_created']
@@ -54,16 +73,6 @@ class TerminalSerializer(BulkModelSerializer):
else:
raise serializers.ValidationError(_('Not found'))
@staticmethod
def get_session_online(obj):
return Session.objects.filter(terminal=obj, is_finished=False).count()
class StatusSerializer(serializers.ModelSerializer):
class Meta:
fields = ['id', 'terminal']
model = Status
class TaskSerializer(BulkModelSerializer):
class Meta:

View File

@@ -29,7 +29,7 @@ logger = get_task_logger(__name__)
@after_app_ready_start
@after_app_shutdown_clean_periodic
def delete_terminal_status_period():
yesterday = timezone.now() - datetime.timedelta(days=1)
yesterday = timezone.now() - datetime.timedelta(days=7)
Status.objects.filter(date_created__lt=yesterday).delete()

View File

@@ -35,7 +35,6 @@ urlpatterns = [
path('command-storages/<uuid:pk>/test-connective/', api.CommandStorageTestConnectiveApi.as_view(), name='command-storage-test-connective'),
# components
path('components/metrics/', api.ComponentsMetricsAPIView.as_view(), name='components-metrics'),
path('components/state/', api.ComponentsStateAPIView.as_view(), name='components-state'),
# v2: get session's replay
# path('v2/sessions/<uuid:pk>/replay/',
# api.SessionReplayV2ViewSet.as_view({'get': 'retrieve'}),

View File

@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
#
import os
from itertools import groupby
from django.conf import settings
from django.core.files.storage import default_storage
@@ -10,9 +11,7 @@ import jms_storage
from common.tasks import send_mail_async
from common.utils import get_logger, reverse
from settings.models import Setting
from . import const
from .models import ReplayStorage, Session, Command
logger = get_logger(__name__)
@@ -141,23 +140,73 @@ def send_command_execution_alert_mail(command):
send_mail_async.delay(subject, message, recipient_list, html_message=message)
class ComponentsMetricsUtil(object):
class ComputeStatUtil:
# system status
@staticmethod
def get_components(tp=None):
def _common_compute_system_status(value, thresholds):
if thresholds[0] <= value <= thresholds[1]:
return const.ComponentStatusChoices.normal.value
elif thresholds[1] < value <= thresholds[2]:
return const.ComponentStatusChoices.high.value
else:
return const.ComponentStatusChoices.critical.value
@classmethod
def _compute_system_stat_status(cls, stat):
system_stat_thresholds_mapper = {
'cpu_load': [0, 5, 20],
'memory_used': [0, 85, 95],
'disk_used': [0, 80, 99]
}
system_status = {}
for stat_key, thresholds in system_stat_thresholds_mapper.items():
stat_value = getattr(stat, stat_key)
if stat_value is None:
msg = 'stat: {}, stat_key: {}, stat_value: {}'
logger.debug(msg.format(stat, stat_key, stat_value))
stat_value = 0
status = cls._common_compute_system_status(stat_value, thresholds)
system_status[stat_key] = status
return system_status
@classmethod
def compute_component_status(cls, stat):
if not stat:
return const.ComponentStatusChoices.offline
system_status_values = cls._compute_system_stat_status(stat).values()
if const.ComponentStatusChoices.critical in system_status_values:
return const.ComponentStatusChoices.critical
elif const.ComponentStatusChoices.high in system_status_values:
return const.ComponentStatusChoices.high
else:
return const.ComponentStatusChoices.normal
class TypedComponentsStatusMetricsUtil(object):
def __init__(self):
self.components = []
self.grouped_components = []
self.get_components()
def get_components(self):
from .models import Terminal
components = Terminal.objects.filter(is_deleted=False).order_by('type')
if tp:
components = components.filter(type=tp)
return components
grouped_components = groupby(components, lambda c: c.type)
grouped_components = [(i[0], list(i[1])) for i in grouped_components]
self.grouped_components = grouped_components
self.components = components
def get_metrics(self, tp=None):
components = self.get_components(tp)
total_count = normal_count = high_count = critical_count = offline_count = \
session_active_total = 0
for component in components:
total_count += 1
if component.is_alive:
def get_metrics(self):
metrics = []
for _tp, components in self.grouped_components:
normal_count = high_count = critical_count = 0
total_count = offline_count = session_online_total = 0
for component in components:
total_count += 1
if not component.is_alive:
offline_count += 1
continue
if component.is_normal:
normal_count += 1
elif component.is_high:
@@ -165,20 +214,23 @@ class ComponentsMetricsUtil(object):
else:
# critical
critical_count += 1
session_active_total += component.state.get('session_active_count', 0)
else:
offline_count += 1
return {
'total': total_count,
'normal': normal_count,
'high': high_count,
'critical': critical_count,
'offline': offline_count,
'session_active': session_active_total
}
session_online_total += component.get_online_session_count()
metrics.append({
'total': total_count,
'normal': normal_count,
'high': high_count,
'critical': critical_count,
'offline': offline_count,
'session_active': session_online_total,
'type': _tp,
})
return metrics
class ComponentsPrometheusMetricsUtil(ComponentsMetricsUtil):
class ComponentsPrometheusMetricsUtil(TypedComponentsStatusMetricsUtil):
def __init__(self):
super().__init__()
self.metrics = self.get_metrics()
@staticmethod
def convert_status_metrics(metrics):
@@ -190,50 +242,74 @@ class ComponentsPrometheusMetricsUtil(ComponentsMetricsUtil):
'offline': metrics['offline']
}
def get_prometheus_metrics_text(self):
def get_component_status_metrics(self):
prometheus_metrics = list()
# 各组件状态个数汇总
prometheus_metrics.append('# JumpServer 各组件状态个数汇总')
status_metric_text = 'jumpserver_components_status_total{component_type="%s", status="%s"} %s'
for tp in const.TerminalTypeChoices.types():
for metric in self.metrics:
tp = metric['type']
prometheus_metrics.append(f'## 组件: {tp}')
metrics_tp = self.get_metrics(tp)
status_metrics = self.convert_status_metrics(metrics_tp)
status_metrics = self.convert_status_metrics(metric)
for status, value in status_metrics.items():
metric_text = status_metric_text % (tp, status, value)
prometheus_metrics.append(metric_text)
return prometheus_metrics
prometheus_metrics.append('\n')
def get_component_session_metrics(self):
prometheus_metrics = list()
# 各组件在线会话数汇总
prometheus_metrics.append('# JumpServer 各组件在线会话数汇总')
session_active_metric_text = 'jumpserver_components_session_active_total{component_type="%s"} %s'
for tp in const.TerminalTypeChoices.types():
for metric in self.metrics:
tp = metric['type']
prometheus_metrics.append(f'## 组件: {tp}')
metrics_tp = self.get_metrics(tp)
metric_text = session_active_metric_text % (tp, metrics_tp['session_active'])
metric_text = session_active_metric_text % (tp, metric['session_active'])
prometheus_metrics.append(metric_text)
return prometheus_metrics
prometheus_metrics.append('\n')
def get_component_stat_metrics(self):
prometheus_metrics = list()
# 各组件节点指标
prometheus_metrics.append('# JumpServer 各组件一些指标')
state_metric_text = 'jumpserver_components_%s{component_type="%s", component="%s"} %s'
states = [
stats_key = [
'cpu_load', 'memory_used', 'disk_used', 'session_online'
]
old_stats_key = [
'system_cpu_load_1', 'system_memory_used_percent',
'system_disk_used_percent', 'session_active_count'
]
for state in states:
prometheus_metrics.append(f'## 指标: {state}')
components = self.get_components()
for component in components:
old_stats_key_mapper = dict(zip(stats_key, old_stats_key))
for stat_key in stats_key:
prometheus_metrics.append(f'## 指标: {stat_key}')
for component in self.components:
if not component.is_alive:
continue
component_stat = component.latest_stat
if not component_stat:
continue
metric_text = state_metric_text % (
state, component.type, component.name, component.state.get(state)
stat_key, component.type, component.name, getattr(component_stat, stat_key)
)
prometheus_metrics.append(metric_text)
old_stat_key = old_stats_key_mapper.get(stat_key)
old_metric_text = state_metric_text % (
old_stat_key, component.type, component.name, getattr(component_stat, stat_key)
)
prometheus_metrics.append(old_metric_text)
return prometheus_metrics
def get_prometheus_metrics_text(self):
prometheus_metrics = list()
for method in [
self.get_component_status_metrics,
self.get_component_session_metrics,
self.get_component_stat_metrics
]:
prometheus_metrics.extend(method())
prometheus_metrics.append('\n')
prometheus_metrics_text = '\n'.join(prometheus_metrics)
return prometheus_metrics_text

View File

@@ -1,8 +1,8 @@
# ~*~ coding: utf-8 ~*~
from django.core.cache import cache
from collections import defaultdict
from django.utils.translation import ugettext as _
from rest_framework.decorators import action
from django.conf import settings
from rest_framework import generics
from rest_framework.response import Response
from rest_framework_bulk import BulkModelViewSet
@@ -16,7 +16,7 @@ from common.mixins import CommonApiMixin
from common.utils import get_logger
from orgs.utils import current_org
from orgs.models import ROLE as ORG_ROLE, OrganizationMember
from users.utils import send_reset_mfa_mail
from users.utils import send_reset_mfa_mail, LoginBlockUtil, MFABlockUtils
from .. import serializers
from ..serializers import UserSerializer, UserRetrieveSerializer, MiniUserSerializer, InviteSerializer
from .mixins import UserQuerysetMixin
@@ -155,10 +155,17 @@ class UserViewSet(CommonApiMixin, UserQuerysetMixin, BulkModelViewSet):
serializer = serializer_cls(data=data, many=True)
serializer.is_valid(raise_exception=True)
validated_data = serializer.validated_data
users_by_role = defaultdict(list)
for i in validated_data:
i['org_id'] = current_org.org_id()
relations = [OrganizationMember(**i) for i in validated_data]
OrganizationMember.objects.bulk_create(relations, ignore_conflicts=True)
users_by_role[i['role']].append(i['user'])
OrganizationMember.objects.add_users_by_role(
current_org,
users=users_by_role[ORG_ROLE.USER],
admins=users_by_role[ORG_ROLE.ADMIN],
auditors=users_by_role[ORG_ROLE.AUDITOR]
)
return Response(serializer.data, status=201)
@action(methods=['post'], detail=True, permission_classes=(IsOrgAdmin,))
@@ -190,16 +197,12 @@ class UserChangePasswordApi(UserQuerysetMixin, generics.RetrieveUpdateAPIView):
class UserUnblockPKApi(UserQuerysetMixin, generics.UpdateAPIView):
permission_classes = (IsOrgAdmin,)
serializer_class = serializers.UserSerializer
key_prefix_limit = "_LOGIN_LIMIT_{}_{}"
key_prefix_block = "_LOGIN_BLOCK_{}"
def perform_update(self, serializer):
user = self.get_object()
username = user.username if user else ''
key_limit = self.key_prefix_limit.format(username, '*')
key_block = self.key_prefix_block.format(username)
cache.delete_pattern(key_limit)
cache.delete(key_block)
LoginBlockUtil.unblock_user(username)
MFABlockUtils.unblock_user(username)
class UserResetOTPApi(UserQuerysetMixin, generics.RetrieveAPIView):

View File

@@ -667,12 +667,20 @@ class User(AuthMixin, TokenMixin, RoleMixin, MFAMixin, AbstractUser):
else:
return user_default
def unblock_login(self):
from users.utils import LoginBlockUtil, MFABlockUtils
LoginBlockUtil.unblock_user(self.username)
MFABlockUtils.unblock_user(self.username)
@property
def login_blocked(self):
key_prefix_block = "_LOGIN_BLOCK_{}"
key_block = key_prefix_block.format(self.username)
blocked = bool(cache.get(key_block))
return blocked
from users.utils import LoginBlockUtil, MFABlockUtils
if LoginBlockUtil.is_user_block(self.username):
return True
if MFABlockUtils.is_user_block(self.username):
return True
return False
def delete(self, using=None, keep_parents=False):
if self.pk == 1 or self.username == 'admin':

View File

@@ -322,50 +322,80 @@ def check_password_rules(password):
return bool(match_obj)
key_prefix_limit = "_LOGIN_LIMIT_{}_{}"
key_prefix_block = "_LOGIN_BLOCK_{}"
class BlockUtil:
BLOCK_KEY_TMPL: str
def __init__(self, username):
self.block_key = self.BLOCK_KEY_TMPL.format(username)
self.key_ttl = int(settings.SECURITY_LOGIN_LIMIT_TIME) * 60
def block(self):
cache.set(self.block_key, True, self.key_ttl)
def is_block(self):
return bool(cache.get(self.block_key))
# def increase_login_failed_count(key_limit, key_block):
def increase_login_failed_count(username, ip):
key_limit = key_prefix_limit.format(username, ip)
count = cache.get(key_limit)
count = count + 1 if count else 1
class BlockUtilBase:
LIMIT_KEY_TMPL: str
BLOCK_KEY_TMPL: str
limit_time = settings.SECURITY_LOGIN_LIMIT_TIME
cache.set(key_limit, count, int(limit_time)*60)
def __init__(self, username, ip):
self.username = username
self.ip = ip
self.limit_key = self.LIMIT_KEY_TMPL.format(username, ip)
self.block_key = self.BLOCK_KEY_TMPL.format(username)
self.key_ttl = int(settings.SECURITY_LOGIN_LIMIT_TIME) * 60
def get_remainder_times(self):
times_up = settings.SECURITY_LOGIN_LIMIT_COUNT
times_failed = self.get_failed_count()
times_remainder = int(times_up) - int(times_failed)
return times_remainder
def incr_failed_count(self):
limit_key = self.limit_key
count = cache.get(limit_key, 0)
count += 1
cache.set(limit_key, count, self.key_ttl)
limit_count = settings.SECURITY_LOGIN_LIMIT_COUNT
if count >= limit_count:
cache.set(self.block_key, True, self.key_ttl)
def get_failed_count(self):
count = cache.get(self.limit_key, 0)
return count
def clean_failed_count(self):
cache.delete(self.limit_key)
cache.delete(self.block_key)
@classmethod
def unblock_user(cls, username):
key_limit = cls.LIMIT_KEY_TMPL.format(username, '*')
key_block = cls.BLOCK_KEY_TMPL.format(username)
# Redis 尽量不要用通配
cache.delete_pattern(key_limit)
cache.delete(key_block)
@classmethod
def is_user_block(cls, username):
block_key = cls.BLOCK_KEY_TMPL.format(username)
return bool(cache.get(block_key))
def is_block(self):
return bool(cache.get(self.block_key))
def get_login_failed_count(username, ip):
key_limit = key_prefix_limit.format(username, ip)
count = cache.get(key_limit, 0)
return count
class LoginBlockUtil(BlockUtilBase):
LIMIT_KEY_TMPL = "_LOGIN_LIMIT_{}_{}"
BLOCK_KEY_TMPL = "_LOGIN_BLOCK_{}"
def clean_failed_count(username, ip):
key_limit = key_prefix_limit.format(username, ip)
key_block = key_prefix_block.format(username)
cache.delete(key_limit)
cache.delete(key_block)
def is_block_login(username, ip):
count = get_login_failed_count(username, ip)
key_block = key_prefix_block.format(username)
limit_count = settings.SECURITY_LOGIN_LIMIT_COUNT
limit_time = settings.SECURITY_LOGIN_LIMIT_TIME
if count >= limit_count:
cache.set(key_block, 1, int(limit_time)*60)
if count and count >= limit_count:
return True
def is_need_unblock(key_block):
if not cache.get(key_block):
return False
return True
class MFABlockUtils(BlockUtilBase):
LIMIT_KEY_TMPL = "_MFA_LIMIT_{}_{}"
BLOCK_KEY_TMPL = "_MFA_BLOCK_{}"
def construct_user_email(username, email):

View File

@@ -11,9 +11,10 @@ from django.contrib.auth import logout as auth_logout
from common.utils import get_logger
from common.permissions import (
PermissionsMixin, IsValidUser,
IsValidUser,
UserCanUpdatePassword
)
from common.mixins.views import PermissionsMixin
from ... import forms
from ...models import User
from ...utils import (

View File

@@ -8,9 +8,10 @@ from django.views.generic.edit import UpdateView
from common.utils import get_logger, ssh_key_gen
from common.permissions import (
PermissionsMixin, IsValidUser,
IsValidUser,
UserCanUpdateSSHKey,
)
from common.mixins.views import PermissionsMixin
from ... import forms
from ...models import User

View File

@@ -13,7 +13,8 @@ from formtools.wizard.views import SessionWizardView
from django.views.generic import FormView
from common.utils import get_object_or_none
from common.permissions import PermissionsMixin, IsValidUser
from common.permissions import IsValidUser
from common.mixins.views import PermissionsMixin
from ...models import User
from ...utils import (
send_reset_password_mail, get_password_check_rules, check_password_rules,

View File

@@ -122,7 +122,7 @@ REDIS_PORT: 6379
# USER_LOGIN_SINGLE_MACHINE_ENABLED: False
#
# 启用定时任务
# PERIOD_TASK_ENABLE: True
# PERIOD_TASK_ENABLED: True
#
# 启用二次复合认证配置
# LOGIN_CONFIRM_ENABLE: False

9
jms
View File

@@ -97,6 +97,14 @@ def check_migrations():
# sys.exit(1)
def expire_caches():
apps_dir = os.path.join(BASE_DIR, 'apps')
code = subprocess.call("python manage.py expire_caches", shell=True, cwd=apps_dir)
if code == 1:
return
def perform_db_migrate():
logging.info("Check database structure change ...")
os.chdir(os.path.join(BASE_DIR, 'apps'))
@@ -116,6 +124,7 @@ def prepare():
check_database_connection()
check_migrations()
upgrade_db()
expire_caches()
def check_pid(pid):

View File

@@ -12,9 +12,9 @@ chardet==3.0.4
configparser==3.5.0
coreapi==2.3.3
coreschema==0.0.4
cryptography==3.2
cryptography==3.3.2
decorator==4.1.2
Django==3.1
Django==3.1.6
django-auth-ldap==2.2.0
django-bootstrap3==14.2.0
django-celery-beat==2.0
@@ -39,7 +39,7 @@ gunicorn==19.9.0
idna==2.6
itsdangerous==0.24
itypes==1.1.0
Jinja2==2.10.1
Jinja2==2.11.3
jmespath==0.9.3
kombu==4.6.8
ldap3==2.4
@@ -49,7 +49,7 @@ olefile==0.44
openapi-codec==1.3.2
paramiko==2.7.2
passlib==1.7.1
Pillow==7.1.0
Pillow==8.1.1
pyasn1==0.4.8
pycparser==2.19
pycryptodome==3.10.1