AI + X 更新日志 - shuai

11.28

1. datasets 页面添加 reload 功能，更新数据集信息

• app/templates/datasets.html：
  • add reload 按钮，表格 add reload 字段，提供只更新选中数据集功能
• app/static/js/dataset.js
  • add reload_dataset()，更新表格数据集信息
• app.datasets.py：
  • add reload_dataset()，对应后台功能：全部更新 / 更新某个
• app.db.py：
  • 更改数据集列表中 type 字段：taskType -> taskRules: defaultShape，因为 dataturks 中 Point, Rectangle, Polygon 共用 taskType IMAGE_POLYGON_BOUNDING_BOX_V2，添加任务映射
  • add clear_dataset()，方便 reload datasets

task_type = {
    'rectangle': 'Detection',
    'polygon': 'Segmentaion'
}

def parse_projects(row):
    taskRules = json.loads(row[3])  # str->dict, 中文
    cats = taskRules['tags'].replace(' ', '').split(',')
    task = task_type[taskRules['defaultShape']]
    return {
        'id': row[0],
        'name': row[1],
        'taskType': task,  # change ori task typye here
        'classes': len(cats),
        'cats': cats,
    }

---

2. 新任务数据集：rotated bbox

• utils.box_utils.py
  • add cvt_poly_fpts_to_center_xywh_angle()，将 dataturks 标注的 多边形 polygon 转换为外界矩形，格式：(x,y,w,h), angle
  • 修改部分函数名称，4pts -> rect
• utils.dataset_utils.py
  • add filter_cigars()，从 dataturks 标注的 annotation 字段，过滤出香烟标注
  • add create_dataset_from_dataturks_json()，从 dataturks 导出的 json 文件中创建数据集
  • update convert_to_coco()，为 ann 新添 angle 字段

use create_dataset_from_dataturks_json()

def build_top_k_dataset(dataset, dataset_dir, top_k=None):
    """ create dataset with top_k classes """
    filted_cats, filted_cats_num, train_num, val_num, test_num = split_and_save_coco_dataset(dataset, dataset_dir=dataset_dir, top_k=top_k)
    prefix = '{}_'.format(top_k) if top_k else ''
    data_cfg = {
        'name': 'Cigar Rotated Box',
        'cats_num': filted_cats_num,
        'classses': len(filted_cats),
        'train': train_num,
        'valid': val_num,
        'test': test_num
    }
    dataset_name = os.path.basename(dataset_dir)
    dump_json(data_cfg, out_path=os.path.join(dataset_dir, prefix + '{}_cfg.json'.format(dataset_name)))


def build_rbox_dataset():
    """ create mulit rbox dataset with multi top_k classes """
    dataset_name = 'cigar_rbox'
    dataset_dir = os.path.join('data', dataset_name)
    in_path = os.path.join(dataset_dir, 'Retail Cigar Dataset.json')
    dataset = create_dataset_from_dataturks_json(dataturks_json_path=in_path)
    # build with top k classes
    top_ks = 10, 20, 40
    for tk in top_ks:
        build_top_k_dataset(dataset, dataset_dir, top_k=tk)
    # build with all classes
    build_top_k_dataset(dataset, dataset_dir)

update convert_to_coco()

# add anns
for ann in anns:
    label = ann['label'][0]
    # add shape judgement
    rect_box, rect_angle = [], 0
    if ann['shape'] == 'rectangle':
        rect_box, rect_angle = cvt_rect_fpts_to_xywh(ann['points'], img_w, img_h), 0
    elif ann['shape'] == 'polygon':
        rect_box, rect_angle = cvt_poly_fpts_to_center_xywh_angle(ann['points'], img_w, img_h)
    anno_coco = {
        "segmentation": [],
        "area": [],
        "iscrowd": 0,
        "image_id": result['img_id'],
        "bbox": rect_box,
        "angle": rect_angle,
        "category_id": categories[label],
        "id": ann_id
    }