沉石鱼惊旋 @ 2022-02-27 14:08:40
rt，有没有创建代码，能把AT的abc的AB题的标题都爬下来，一个一个复制太累了，差不多五百个呢……
by YCSluogu @ 2022-02-27 14:28:59
你可以在python中用selenium配合xpath来爬取
by tribool4_in @ 2022-02-27 14:32:11
@chenyuxuan__2009
import urllib.request
import re
url = 'https://atcoder.jp/contests/archive?ratedType=1&page='
rep = re.compile('<a href="/contests/(abc.+?)">')
abcs = ['abc241', 'abc240', 'abc239', 'abc238', 'abc237', 'abc236', 'abc235', 'abc234', 'abc233', 'abc232', 'abc231', 'abc230', 'abc229', 'abc228', 'abc227', 'abc226', 'abc225', 'abc224', 'abc223', 'abc222', 'abc221', 'abc220', 'abc219', 'abc218', 'abc217', 'abc216', 'abc215', 'abc214', 'abc213', 'abc212', 'abc211', 'abc210', 'abc209', 'abc208', 'abc207', 'abc206', 'abc205', 'abc204', 'abc203', 'abc202', 'abc201', 'abc200', 'abc199', 'abc198', 'abc197', 'abc196', 'abc195', 'abc194', 'abc193', 'abc192', 'abc191', 'abc190', 'abc189', 'abc188', 'abc187', 'abc186', 'abc185', 'abc184', 'abc183', 'abc182', 'abc181', 'abc180', 'abc179', 'abc178', 'abc177', 'abc176', 'abc175', 'abc174', 'abc173', 'abc172', 'abc171', 'abc170', 'abc169', 'abc168', 'abc167', 'abc166', 'abc165', 'abc164', 'abc163', 'abc162', 'abc161', 'abc160', 'abc159', 'abc158', 'abc157', 'abc156', 'abc155', 'abc154', 'abc153', 'abc152', 'abc151', 'abc150', 'abc149', 'abc148', 'abc147', 'abc146', 'abc145', 'abc144', 'abc143', 'abc142', 'abc141', 'abc140', 'abc139', 'abc138', 'abc137', 'abc136', 'abc135', 'abc134', 'abc133', 'abc132', 'abc131', 'abc130', 'abc129', 'abc128', 'abc127', 'abc126', 'abc125', 'abc124', 'abc123', 'abc122', 'abc121', 'abc120', 'abc119', 'abc118', 'abc117', 'abc116', 'abc115', 'abc114', 'abc113', 'abc112', 'abc111', 'abc110', 'abc109', 'abc108', 'abc107', 'abc106', 'abc105', 'abc104', 'abc103', 'abc102', 'abc101', 'abc100', 'abc099', 'abc098', 'abc097', 'abc096', 'abc095', 'abc094', 'abc093', 'abc092', 'abc091', 'abc090', 'abc089', 'abc088', 'abc087', 'abc086', 'abc085', 'abc084', 'abc083', 'abc082', 'abc081', 'abc080', 'abc079', 'abc078', 'abc077', 'abc076', 'abc075', 'abc074', 'abc073', 'abc072', 'abc071', 'abc070', 'abc069', 'abc068', 'abc067', 'abc066', 'abc065', 'abc064', 'abc063', 'abc062', 'abc061', 'abc060', 'abc059', 'abc058', 'abc057', 'abc056', 'abc055', 'abc054', 'abc053', 'abc052', 'abc051', 'abc050', 'abc049', 'abc048', 'abc047', 'abc046', 'abc045', 'abc044', 'abc043', 'abc042']
# for i in range(1, 6):
# req = urllib.request.Request(url + str(i), method='GET') #, headers=headers
# res = urllib.request.urlopen(req)
# html = res.read().decode('utf8')
# abcs += rep.findall(html)
# print(abcs)
for abcid in abcs:
res = urllib.request.urlopen('https://atcoder.jp/contests/' + abcid + '/tasks')
html = res.read().decode('utf8')
rep2 = re.compile('<td><a href="/contests/' + abcid + '/tasks/.+?">(.+?)</a></td>')
print(rep2.findall(html)[:2])by piggy123 @ 2022-02-27 14:33:24
@chenyuxuan__2009 https://www.luogu.com.cn/paste/4gtb8dhi
by 沉石鱼惊旋 @ 2022-02-27 14:33:40
@wangwls %%%tql
by 沉石鱼惊旋 @ 2022-02-27 14:34:18
@piggy123 谢谢
by tribool4_in @ 2022-02-27 14:34:57
@chenyuxuan__2009 要一段时间才能爬完（）
我现在才爬下来两页
by tribool4_in @ 2022-02-27 14:35:33
额正则表达式多是件美事为啥要用bs4（）
by 沉石鱼惊旋 @ 2022-02-27 14:36:07
@wangwls dalao方便私信吗
by piggy123 @ 2022-02-27 14:42:44
@wangwls bs4好用啊，如果你发现一个问题可以用正则解决，那么他就寄了
by tribool4_in @ 2022-02-27 14:44:26
我是 懒得用第三方库癌 患者/ll