Papers

Research papers turned into pages you can build from: the idea in plain words, the method step by step, every hyperparameter, the results, and where the code differs from the paper.

1653 pages

Paper · 2026-10-20Reinforcement learning for hardware acceleration and system-level optimization on FPGA and integrated-circuit platforms: A surveySurvey of more than 100 studies running reinforcement learning on FPGAs and integrated circuits. It sorts them by RL family (MDP, Q-learning, SARSA, DQN, DDPG, actor-critic, Monte Carlo, PPO, multi-agent), by RL role (workload or optimizer) and by application. No new experiments; the reported best results, such as 1400x for one Q-learning design, are author-reported.Paper · 2026-10-14Object Pose from 2-D to 3-D Point and Line CorrespondencesEstimates the rotation and translation of a camera relative to a 3-D object from point and/or line correspondences. Each correspondence gives quadratic constraints on a dual-number quaternion; a penalised sum of squares is minimised with a trust-region method. With 100 points and 1 px noise, author-reported translation error is 0.0002 cm against 11.48 cm for the linear Faugeras-Toscani method.Paper · 2026-10-13Calibrated Disambiguation for Partial Multi-label LearningPML-CD trains a multi-label image classifier on over-annotated labels by weighting each candidate positive with a calibrator. The calibrator maps a 20-bin histogram of confidences on candidate positives to per-bin reliability, and an extra prototype-cosine loss is added. Author-reported mAP beats nine baselines in 15 of 16 settings on VOC, COCO, NUS-WIDE and CUB at 20-80% noise.Paper · 2026-10-12Boosting the Learning for Ranking PatternsLearns a user's ranking of mined patterns as weights over interestingness measures. Pairwise measure gaps against user rankings feed an AHP eigenvector solve. An active mode asks pairwise questions chosen by a sensitivity heuristic. On association rules (author-reported) it is far faster than an SVM ranker and more accurate in active mode, but less accurate in passive mode.Paper · 2026-10-12Learning to Play: Reinforcement Learning and GamesTextbook on reinforcement learning in games. Covers MDPs, heuristic search, Monte Carlo tree search with UCT, deep RL and self-play, using AlphaGo, AlphaGo Zero and AlphaZero as case studies. Gives the MCTS loop and the reported AlphaGo settings. Author-reported: AlphaGo beat Fan Hui, Lee Sedol and Ke Jie; AlphaZero learned Go, Chess and Shogi from self-play.Paper · 2026-10-11Data Portability Revisited: Toward the Human-Centric, AI-Driven Data Ecosystems of TomorrowA legal article arguing that EU data-portability rules (GDPR Art. 20, Data Act, DMA, EHDS, AI Act) mainly serve service providers. It compares three architectures (consent and opt-out, enterprise-centric pods and GAIA-X, user-held private-by-default data) and urges treating portability as a fundamental right. It has no experiments or code; its only numbers are regulatory parameters.Paper · 2026-10-05Virtual model control for compliant reaching under uncertaintiesBuilds reaching controllers for a force-controlled arm from virtual springs and dampers, mapped to joint torques through Jacobians, with no inverse kinematics. Six designs cover simple, moving-reference and force-driven reaching, plus obstacle avoidance. A human blocking the hand on a Sciurus17 humanoid drew up to 4.53 N, versus 35.77 N for an IK-based PID baseline (author-reported).Paper · 2026-10-02Sample complexity of variance-reduced policy gradient: weaker assumptions and lower boundsProposes DEF-PG, a variance-reduced policy gradient using defensive importance sampling, which keeps importance weights bounded without a uniform variance assumption. Under bounded rewards and regular policy scores it reaches O(eps^-3) first-order sample complexity. It also proves black-box lower bounds of Omega(eps^-4) for one-policy and Omega(eps^-3) for coupled feedback.Paper · 2026-10-01The Impact of Processing Parameters on High-Accuracy Measurements in UAV PhotogrammetryFull-factorial study of 768 Bundle Block Adjustment (BBA) processing variants in Agisoft Metashape on ten UAV surveys (220 ha). Number of GCPs, extra camera-calibration corrections and PPK georeferencing matter most. Best variant: 16 mm 3D RMSE at check points, worst 303 mm. Tuned settings halve the systematic vertical error (14 to 7 mm). Author-reported.Paper · 2026-09-30Recognition of Urbanized Areas in UAV-Derived Very-High-Resolution Visible-Light ImageryCompares 16 RGB and CIELab vegetation indices, thresholded on one UAV dataset, with linear, MLP and CNN pixel classifiers for urbanized vs non-urbanized areas in UAV RGB orthomosaics. Tested on a second site with another drone and season. Best network: MCC about 0.82, accuracy about 0.96. Best index, Excess Blue: MCC about 0.61, accuracy about 0.87. Author-reported.Paper · 2026-09-29Learning from synthetic photorealistic raindrop for single image raindrop removalRenders a physics-based synthetic dataset of adherent raindrops on Cityscapes images, with pixel-level masks, and trains a three-stage network: raindrop mask detection, region reconstruction, and refinement. Author-reported PSNR 41.29 dB and SSIM 0.9921 on the synthetic test set, above the compared baselines.Paper · 2026-09-28Word Similarity Datasets for Indian Languages: Annotation and Baseline SystemsBuilds manually annotated word-similarity test sets for six Indian languages (Urdu, Telugu, Marathi, Punjabi, Tamil, Gujarati) by translating and re-annotating RG-65 and WordSim-353, then scores Skip-gram, CBOW and FastText embeddings with Spearman's rho on three of them. FastText with an OOV fallback reaches 45.5 (Urdu), 38.7 (Marathi) and 46.0 (Telugu). Author-reported.Paper · 2026-09-27Showing versus Doing: Teaching by DemonstrationModels teaching by demonstration as Bayesian pedagogy: a teacher picks trajectories that best reveal the goal or reward to an observer who assumes a plain planner. Two human studies (goal grid, coloured-tile rewards) show that showing differs from doing in path choice, and that inverse RL learns the reward better from showing trajectories. Author-reported. The released code has no teacher model.Paper · 2026-09-22A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language ModelEstimates the full carbon footprint of building Noor, a 1.3B to 13B Arabic LLM family: data storage and transfer, R&D, pretraining, staff flights and projected inference. Electricity is 59.1 MWh and emissions 36.5 t CO2e, 65% from training. Location of compute, not model size alone, drives most of it. Author-reported.Paper · 2026-09-21End-to-end Jordanian dialect speech-to-text self-supervised learning frameworkWav2Vec 2.0-style self-supervised pre-training on unlabelled Jordanian Arabic speech, then five noisy-student generations of pseudo-labelling and augmentation on the authors' own ~97k-recording set. Word error rate falls from 0.568 (teacher) to 0.515 at generation five on their evaluation set. Author-reported, single run. No official code found.Paper · 2026-09-20A multi-temporal dataset for mapping burned areas in the Brazilian Cerrado using time series of remote sensing imageryBuilds a 2020-2022 tabular dataset of WFI satellite time series (blue, green, red, NIR and five spectral indices) on a 500 m grid over Chapada dos Veadeiros National Park, Brazil, with cells hand-labelled as totally burned, partially burned or non-burned. A Random Forest reaches 0.94-0.95 F1 on the best binary setup; burned-area IoU is only 0.49-0.71 for the best validation setup. Author-reported.Paper · 2026-09-15High-Fidelity Digital Twin Data Models by Randomized Dynamic Mode Decomposition and Deep Learning with Applications in Fluid DynamicsPicks the number of modes in dynamic mode decomposition with a randomized SVD and an error-minimising rank search, then fits a cascade-forward network (NLARX) to the mode amplitudes to form a digital twin data model. On three viscous Burgers shock cases (Re 10^2 to 10^4) the author reports DMD errors of 5.4e-8 to 4.4e-7 and group-level DTM errors of 1.3e-4 to 8.1e-4, all in-sample.Paper · 2026-09-14Can AI systems have free will?Philosophical essay proposing a pragmatic test for free will in AI. Ask whether the system is best explained as an intentional agent with alternative possibilities for choice and causal control over its actions, not whether it has a mysterious property, is indeterministic or is unpredictable. If so, it has free will in a practical sense. No experiments; the verdict is conditional.Paper · 2026-09-13A panoramic aerodynamic performance prediction method for turbomachinery cascades using transformer-enhanced neural operatorPredicts the basic flow fields of a transonic compressor (pressure, temperature, squared velocities, mass flux) from 28 blade-geometry variables with a transformer-enhanced neural operator (TNO). Performance metrics are computed from the predicted fields. Author-reported on Rotor 37: mean field error 0.083% vs 0.572% for FNO.Paper · 2026-09-11MAAPO:an innovative membrane algorithm based on artificial protozoa optimizer for multilevel threshold image segmentationAdds membrane-style splitting and merging of the population, triggered by a volume diversity metric, and roulette fitness-distance selection of the reference point to the artificial protozoa optimizer. Author-reported: best Friedman rank (3.35) of 13 algorithms on 30 CEC2017 functions at 10-D; best average ranks for Kapur-entropy thresholding of three images.Paper · 2026-09-09Precision in Rice Variety Classification using Stacking-Based Ensemble LearningFine-tunes ten ImageNet CNNs on 27,000 augmented images of 20 rice varieties, keeps the four best (EfficientNetV2L, VGG16, MobileNetV2, DenseNet201) and stacks them under 14 meta-learners; XGBoost reaches 100% on the self-curated set. Base models reach 97.5-99.2% validation accuracy. Author-reported; split and leakage details are unclear.Paper · 2026-09-07Impact of canny edge detection preprocessing on performance of machine learning models for Parkinson's disease classificationNine classifiers are trained on spiral drawings from the Parkinson's Image dataset, with and without Canny edge detection plus Hessian filtering and augmentation, using 11 image statistics and feature selection. Canny preprocessing degrades most models. Augmented data gives AUC up to 0.97 (RF, XGBoost, KNN). Author-reported; 10-fold CV, no held-out test set.Paper · 2026-08-21A Resilience Recovery Method for Complex Traffic Network Security Based on Trend ForecastingModels failure spread in a traffic network with an SIRD model plus a per-node risk value (SIRD-R). Defines real-time and overall resilience from network performance, forecasts the resilience curve with an LSTM, and applies protection and recovery on that forecast. Author-reported: LSTM forecast R2 0.96842; overall traffic resilience 0.7261 to 0.9354 with the combined strategy.Paper · 2026-08-17Physics of Agents: Statistical Mechanics Predicts Collective Behavior of AI AgentsAuthor-reported: 32 persona-conditioned LLM agents vote on binary questions and exchange messages over signed social ties. A logistic Ising-style update with separate concordant, discordant and unsigned couplings predicts next votes, reaching 75-86% one-step balanced accuracy on held-out questions and beating the baselines.Paper · 2026-08-17Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical IntelligenceZetta keeps a frozen VLA (pi0.5 on LIBERO-Pro, GR00T N1.5 on RoboCasa) and evolves code-based runtime critics and recovery skills from rollout failures, admitted only after same-seed and held-out checks. Author-reported: RoboCasa 18-task macro 73.56% to 93.56%; LIBERO-Pro 40-pair macro 32.00% to 71.13%. The abstract's 90.8% is not traceable.Paper · 2026-08-14Deep Reinforcement Learning solution for pickup and delivery routing problems with time window and capacity constraintsAdapts the JAMPR attention-based reinforcement-learning route builder to pickup-and-delivery routing with capacity and time windows. Adds a pickup-order mask and a soft penalty (13 per unit distance, 10 per missed client). Author-reported: better early solutions than OR-Tools at every size tested; final quality near OR-Tools only up to 200 customers. Training takes 3 to 336 hours on one GPU.Paper · 2026-08-06MASS: Multiplayer World Models with Authoritative Shared StateMASS keeps one authoritative typed state of a multiplayer world. A learned Logic Engine advances it each tick from all players' actions, and a learned Rendering Engine draws any camera from that state. On a matched two-view Snake benchmark, the authors report parser state recovery of 0.764 against 0.128 for the best video baseline, zero cross-view disagreement, and 1,024-entity rollouts.Paper · 2026-07-28Embodied Agents Take Control: Minimal-Interface Zero-Shot Agents Rival Industrial-Scale Policies in Vision-and-Language NavigationCoding-agent harnesses do zero-shot R2R-CE navigation with a monocular RGB camera, four primitives (forward, two turns, STOP) and no map, memory or trained policy. Replicated default-effort opus-5 runs average 70.7% SR (sd 3.5); fable-5 at max effort reaches 78%. An optional trained waypoint tool gives 76.7% in about half the steps of max-effort primitives. Author-reported, 100 episodes.Paper · 2026-07-27MemTX: Transactional Belief Commit for Stateful Agent MemoryMemTX treats an agent memory write as a staged belief: records pass evidence, validity, conflict and ancestry checks before commit, irreversible tool calls wait while tentative records are in flight, and retracting a belief repairs its descendants by type. Author-reported: 0.956 task success on Qwen3-8B on the authors' 90-case suite, against 0.822 for the best baseline.Paper · 2026-07-21Breaking the Homogeneity Assumption: Specialized Multi-Generator Adversarial Learning for Rare Failure Detection in Predictive MaintenanceTrains one WGAN-GP tabular GAN per failure type on the rare failure rows of the AI4I 2020 dataset, then pools their synthetic failures into the training set. With a Random Forest, PR-AUC is 0.857 and recall 0.882, against 0.835 and 0.588 for cost-sensitive learning (author-reported, single run).Paper · 2026-07-08Single-Rollout Asynchronous Optimization for Agentic Reinforcement LearningSAO trains an LLM with asynchronous RL on one rollout per prompt instead of GRPO's group. It uses rollout log-probs as the behaviour policy, a double-sided token mask, a critic updated twice per step with frozen attention, and a skip-observation GAE. Author-reported: Qwen3-30B-A3B reaches 97.3 on AIME2025 and 29.8 on SWE-Bench Verified, while GRPO collapses near step 160.Paper · 2026-07-07Ad Headline Generation using Self-Critical Masked Language ModelGenerates one short advertising headline for a set of products with a BERT-Large masked language model, then fine-tunes it with self-critical policy gradient, using ROUGE-L F1 against approved headlines as reward. Author-reported: ROUGE-L is 6.33 above a bi-LSTM baseline, and 6.53% more headlines get the top audit rating than human-submitted ones.Paper · 2026-07-06GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy MemoryGEM-Occ turns each posed observation into semantic Gaussians for occupied space and rays for visible free space, then fuses them into a persistent indoor memory with confidence- and visibility-aware updates. Its author-reported results on the HIOcc benchmark: 61.37 local IoU, 46.20 room mIoU, and 0.81 MB/m² of memory against 1.36 for flat Gaussian memory at building scale.Paper · 2026-06-30Exploring the relationship between team institutional composition and novelty in academic papers based on fine-grained knowledge entitiesScores paper novelty as the share of unusual pairs among method, dataset, tool and metric entities extracted from NLP full texts with SciBERT+BiLSTM (86.27 F1). Across 14,812 papers, academic-only and mixed teams score higher than industry-only teams in regressions.Paper · 2026-06-30Improving path-tracking performance of an articulated tractor-trailer system using a non-linear kinematic modelA kinematic model of an articulated tractor with front steering and a central joint, with a hitched trailer, is embedded in an NMPC that tracks the trailer's xy-position. In ROS/Gazebo simulation on a 1600 m2 seeding field, trailer error stays below 16 cm along x after start-up and below 12 cm along y in headland turns (author-reported, simulation only).Paper · 2026-06-29Revealing the Technology Development of Natural Language Processing: A Scientific Entity-Centric PerspectiveExtracts method, dataset, metric and tool entities from full-text ACL, EMNLP and NAACL papers (2000-2022) with a SciBERT+BiLSTM cascade tagger, merges name variants, and scores each entity by a yearly co-occurrence z-score. Author-reported: F1 87.00 on the authors' 311-sentence test split; 179 high-impact new entities (130 methods), with BERT highest at z 43.31.Paper · 2026-06-25Utilizing Cognitive Signals Generated during Human Reading to Enhance Keyphrase Extraction from MicroblogsAdds per-word EEG and eye-tracking features from the ZuCo reading corpus to BiLSTM-based keyphrase taggers for tweets. Author-reported F1, Bi-LSTM baseline to SATT-BiLSTM+CRF with EEG: 61.67 to 67.82 (Election-Trec) and 70.67 to 82.46 (General-Twitter). EEG helps more than eye tracking. Released code differs from the paper.Paper · 2026-06-24Spam and Sentiment Detection in Arabic Tweets Using MARBERT ModelFine-tunes MARBERT, an Arabic BERT-style model pretrained on dialectal and standard Arabic tweets, separately for five-class sentiment and for spam on private Saudi Telecom (STC) customer tweets. Best sentiment F1 is 0.75 (author-reported), reached with batch size 96, learning rate 2e-5 and 5 epochs; the sarcasm and indeterminate classes stay weak. No official code.Paper · 2026-06-22Transfer learning-based method for automated ewaste recycling in smart citiesFine-tunes the output layers of an ImageNet-pretrained AlexNet to sort 12 smartphone classes from 6 brands, using 650 back-side photos with 9x augmentation. SGD with momentum at learning rate 3e-4 reached 98.35% ± 2.02 accuracy over seven trials, ahead of Adam and RMSProp.Paper · 2026-06-19VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action ModelsVLA-FAIL flags failures in finetuned vision-language-action policies without failure data. It alarms when either LLMD, a token-wise Mahalanobis distance on last-layer features under fixed prior noise, or ACC, a velocity-normalized mismatch between overlapping action chunks, passes a calibrated threshold. Author-reported: top three on most tasks; lowest average AUCPDT on Libero-Plus with X-VLA.Paper · 2026-06-18EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action PoliciesEventVLA gives a VLA policy fixed visual anchors (first frame plus frames 15 and 30 steps back) and a Keyframe Evidence Memory head that predicts, for each future action step, whether that frame will be task-critical, then stores those frames. Reported: 67.8% on RMBench, 75.2% on the new RoboTwin-MeM benchmark (18.0% with anchors only), 78.8% on four real bimanual tasks. All author-reported.Paper · 2026-06-16GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?GameCraft-Bench is a benchmark of 140 Godot game-building tasks in 15 families. Agents submit a full project plus replayable input traces; a verifier replays them and a multimodal judge scores the video frames against a hidden rubric. The best of seven agent configurations reaches 41.46%.Paper · 2026-06-06Causal Agent Replay: Counterfactual Attribution for LLM-Agent FailuresCausal Agent Replay (CAR) attributes a failed LLM-agent run to a step by redrawing that step under the same policy, re-running the rest, and comparing outcome distributions with confidence intervals. A Monte-Carlo Shapley estimator shares credit between interacting steps. On synthetic models with known answers it recovers the pivotal step and splits a two-step interaction (0.44, 0.45).Paper · 2026-06-03Towards Accurate Model Selection in Deep Unsupervised Domain AdaptationDeep Embedded Validation (DEV) picks the best unsupervised domain adaptation model without target labels. A domain classifier on the model's features reweights labelled source validation samples into a density-ratio estimate, with a control variate to cut variance. Author-reported on VisDA: DEV selects MCD models at 72.42% mean accuracy, near target-label selection (72.55%).Paper · 2026-06-02Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor FusionOne compact network predicts depth and semantic segmentation for four views, LiDAR segmentation and a bird's-eye-view map in one pass from RGB, DVS and LiDAR inputs. A modified GradNorm adapts the loss weights once per epoch. Author-reported: lower total metric than the combined single-task baselines on three of four test sets, with under 2% of their parameters.Paper · 2026-05-31Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information DecompositionSensory PID decomposes how video and audio contribute to an omni-modal model's answer under a fixed text instruction (unique, redundant, synergistic), estimated training-free with BATCH. On MUSIC-AVQA, unique visual information dominates and synergy stays small. PID-guided LoRA reweighting: author-reported +2.3 pp MMStar on Qwen2.5-VL-7B.Paper · 2026-05-28Evaluation of Conversational Agents: Understanding Culture, Context and Environment in Emotion DetectionEmotion detector for face images and speech, aimed at Black African settings. Three-block CNNs classify facial expressions (FER-2013 plus about 1,000 local images) and speech features (MFCC, chroma, ZCR, RMS, mel spectrogram). An Audio-Frame Mean Expression (AFME) step compares top-3 emotions from both and flags sarcasm when none match. Author-reported 95.7% accuracy for CNN+AFME.Paper · 2026-05-25Electricity Consumption Forecasting: An Approach Using Cooperative Ensemble Learning with SHapley Additive exPlanationsForecasts 12 months of monthly electricity use at two Brazilian university campuses with LSTM, random forest, SVR and XGBoost, tuned by a genetic algorithm and pruned with SHAP. Its Weaker Separator Booster pushes the LSTM forecast away from the mean of the other three. Author-reported sMAPE: 13.90% (Palmas), 18.72% (Coronel Vivida).Paper · 2026-05-19Generative Recursive ReasoningMakes a recursive reasoner probabilistic: each latent refinement step adds learned Gaussian guidance, trained with amortized variational inference, and N sampled trajectories are majority-voted. Author-reported: 96.1% on Sudoku-Extreme at N=20 (TRM 83.6%, single run) and 99.5% on 8x8 N-Queens at N=1.Paper · 2026-05-15CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?CHI-Bench has 75 long-horizon healthcare workflow tasks (prior authorization, utilization review, care management) run in a simulated environment of 20 apps via 87 MCP tools, guided by a 1,279-document handbook. Of 30 agent configurations, the best resolves 28.0% of tasks at pass@1 and none exceeds 20% at pass^3. Author-reported.Paper · 2026-05-12A Transfer Learning Evaluation of Deep Neural Networks for Image ClassificationFine-tunes eleven ImageNet-pretrained CNNs on five image datasets, training either only the classifier or all layers, for one or ten episodes. Scores are accuracy, accuracy density (accuracy / #parameters), training time and model size. Author-reported: SqueezeNet has the highest density under full tuning, ResNet-18 under classifier-only tuning. No numeric results table.Paper · 2026-05-12SafeManip: A Property-Driven Benchmark for Temporal Safety Evaluation in Robotic ManipulationBenchmark that checks recorded manipulation rollouts against reusable LTLf safety rules (eight categories) grounded in simulator state, scoring violations per triggered event. Audits seven policies on RoboCasa365 and LIBERO. Author-reported: higher task success does not imply fewer per-opportunity violations; RoboCasa365 per-trigger risk rises from about 0.03 (atomic) to 0.075 (long) tasks.Paper · 2026-05-11Language Modeling with Hyperspherical FlowsS-FLM is a flow language model whose tokens are unit-norm embeddings on a hypersphere. Generation rotates random points toward token embeddings, trained with cross-entropy on a truncated, adaptive noise schedule with an nGPT-style backbone. Author-reported: GSM8K accuracy rises from under 1% for prior flow LMs to 12.4% (exact velocity) and 18.0% (top-1 velocity) at T=1.Paper · 2026-05-11Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and WhyTraining-free diagnostic that scores, per token and branch node, whether a GKD distillation gradient points toward success. The ideal gradient comes from targeted student rollouts; the cosine alignment is higher on failing rollouts, and the best teacher context depends on student size and task. Author-reported on Qwen3-0.6B/1.7B (BoolQ, MMLU, AIME 2025).Paper · 2026-05-09REAP: Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator for Real2Sim2Real TransferREAP trains a camera-based Soft Actor-Critic parking policy in a 3D Gaussian Splatting simulator, with BEV auxiliary losses, behavior cloning from a Reeds-Shepp planner and soft predictive collision rewards. Author-reported: 83.5% parking success on standard slots and 70.0% on mechanical slots in simulation; 55-85% real-vehicle success.Paper · 2026-05-08KL for a KL: On-Policy Distillation with Control Variate BaselineSubtracts a detached baseline from each token's teacher-student log-ratio reward in on-policy distillation. The baseline is the per-token negative reverse KL, taken from the existing forward pass: no critic, no extra rollouts, gradient still unbiased. A top-k baseline is cheaper. Author-reported: up to +3 average points over OPD on Qwen3-1.7B, matching full-vocabulary OPD.Paper · 2026-05-08SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure ScenariosA live Kubernetes benchmark for SRE agents: 90 problems inject faults across the stack, background noise, and metastable or correlated failures. An LLM checklist grades diagnoses and system-state checks grade mitigation. Author-reported: 38.1-72.6% diagnosis and 40.4-78.5% mitigation across four agent-model pairs; diagnosis drops under noise.Paper · 2026-05-07Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive SurveySurvey of running deep vision models on edge devices. Proposes a low-end, medium-range and high-performance device taxonomy, reviews pruning, sparsity, quantisation, lightweight networks and NAS, and surveys computer-vision and medical applications. Survey only: no new experiments; figures are quoted from cited works, author-reported.Paper · 2026-05-07Grokking or Glitching? How Low-Precision Drives Slingshot Loss SpikesAttributes Slingshot loss spikes in long unregularised training to float32 rounding in softmax cross-entropy. Once the correct-class gradient rounds to zero, the classifier and feature means drift and grow exponentially (Numerical Feature Inflation). Float64 loss, a zero-sum gradient projection or BatchNorm before the classifier removes the spikes. Author-reported on MLP, CNN, ViT and GPT runs.Paper · 2026-05-06Evidence-based anomaly detection in clinical domainsFlags a patient-management decision as anomalous when a Bayesian network or Naive Bayes model, learned from the k past patients most similar to the target (weighted Mahalanobis distance), gives it low predictive probability. Author-reported on 100 PORT pneumonia admission decisions: best AUC-ROC 0.81, and 0.5 precision at 0.53 recall for the BBN model.Paper · 2026-05-06Feature importance analysis for patient management decisionsRanks temporal features of electronic health records (time since last test, last value, trend, nadir) by single-feature AUC for predicting whether a clinician orders a lab or gives a medication within 24 hours, for 4,486 post-surgical cardiac patients. Top-1, 3 and 30 features in a linear SVM reach AUCs of about 70 to 90 percent for 10 labs and 10 medications. No official code.Paper · 2026-05-01Trading off rewards and errors in multi-armed banditsBandit learner that trades cumulative reward against accuracy of every arm's mean. Scores allocations as w times reward minus (1-w) times a scaled error. ForcingBalance forces about eta*sqrt(t) pulls per arm, then tracks the best allocation. Regret is O~(n^-1/2) asymptotically (author-reported). On educational data, reward falls between UCB's and error-only methods'.Paper · 2026-04-30Bayesian policy gradient and actor-critic algorithmsBayesian policy gradient (BPG) models the gradient as a Gaussian process and applies Bayesian quadrature with Fisher kernels, giving closed-form posterior mean, covariance and natural gradient from sampled paths. Bayesian actor-critic (BAC) uses a GPTD critic. Author-reported: fewer samples than Monte-Carlo on a bandit and an LQR; BAC reaches better policies on three control tasks.Paper · 2026-04-29Semi-supervised learning with max-margin graph cutsSemi-supervised classifier: a regularized harmonic function on a k-NN graph labels the unlabeled points, then a kernel max-margin (hinge-loss) classifier is trained only on points whose inferred label is confident. Author-reported: lower test error than manifold-regularized SVMs in 29 of 36 UCI settings.Paper · 2026-04-23Application of Artificial Intelligence for Fraudulent Banking Operations RecognitionCompares seven standard classifiers for detecting fraudulent card transactions on the Kaggle Credit Card Fraud dataset, after random undersampling of the training split and standardization of Time and Amount. Logistic regression is the best single model at AUC 0.946; a stacked ensemble reaches AUC 0.954 (author-reported, no code).Paper · 2026-04-20CAPO: Critic-Guided Action-Aligned Policy Optimization for Advancing LLM Agent CapabilitiesCAPO trains LLM agents with a critic that scores the state before each complete action. The action gets one GAE advantage, and its token ratios are combined into a length-calibrated, batch-centred ratio that is clipped as one unit. Author-reported: it beats PPO, GRPO, GiGPO and ST-PPO on QA, paper search, ALFWorld and WebShop. The released code lacks the ratio.Paper · 2026-04-16Autogenesis: A Self-Evolving Agent ProtocolAutogenesis Protocol (AGP) manages updates to prompts, agents, tools, environments and memory as registered, versioned resources, with a shared propose, assess, commit lifecycle. Its system AGS improves over its non-evolving configuration on GAIA (test 79.07 to 89.04) and on GPQA-Diamond and LeetCode. Author-reported.Paper · 2026-04-14Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic ReasoningNemotron 3 Super: 120.6B-total, 12.7B-active hybrid Mamba-2/attention MoE with LatentMoE experts (512 experts, top-22) and shared-weight MTP heads for speculative decoding. Pretrained in NVFP4 on 25T tokens, then SFT and multi-environment RL. Author-reported: 2.2x and 7.5x throughput vs GPT-OSS-120B and Qwen3.5-122B at 8k/64k; SWE-Bench (OpenHands) 60.47.Paper · 2026-04-13LangFlow: Continuous Diffusion Rivals Discrete in Language ModelingLangFlow is a continuous diffusion language model over token embeddings. It trains with token cross-entropy, samples with a deterministic Euler ODE solver, uses a learned Gumbel noise schedule and self-conditioning, and scores likelihood with an ODE bound. Author-reported: PPL 30.0 on LM1B and 24.6 on OpenWebText, matching masked diffusion.Paper · 2026-04-09KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent EvaluationKnowU-Bench is an online Android benchmark for mobile GUI agents. The agent sees behavioural logs but not the user's hidden profile, may ask clarifying questions answered by an LLM user simulator, and must decide whether to act, ask consent or stay silent. Scores mix rule checks with an LLM judge. Across 11 models, Claude Sonnet 4.6 leads with 60.4% overall success (author-reported).Paper · 2026-04-09Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth TransformersTrains small transformers from scratch on synthetic multi-hop knowledge-graph queries. Reusing one block R times gives systematic generalization via a three-stage grokking process and depth extrapolation when inference-time recurrence is raised. Vanilla 4-layer models fail both; overthinking limits very deep chains.Paper · 2026-04-06MedGemma 1.5 Technical ReportMedGemma 1.5 4B extends the 4B MedGemma vision-language model to 3D CT and MRI volumes (slice sampling, HU-to-RGB windowing), whole-slide pathology (896-px patch sampling), chest X-ray bounding boxes, longitudinal chest X-ray comparison and lab-report extraction. Author-reported: 3D CT accuracy 58.2 to 61.1 and 3D MRI 51.3 to 64.7 over MedGemma 1 4B.Paper · 2026-04-02Backup-Based Safety Filters: A Comparative Review of Backup CBF, Model Predictive Shielding, and gatekeeperReview, not a new method. Puts Backup CBF, Model Predictive Shielding (MPS) and gatekeeper in one safety-filter framework, compares them by filter-inactive sets, and proves MPS is a special case of gatekeeper. Author-reported simulations: gatekeeper keeps the nominal controller on 87.5% of steps in reach-avoid (MPS 55.3%, Backup CBF 28.8%) and on 100% in highway overtake.Paper · 2026-03-30Atomic and Holistic LLM Judges for Reference-Grounded Support Labels: A Prompt-Controlled ComparisonCompares four single-call LLM-judge designs for three-way support labels given a reference: candidate-side claim splitting, a holistic rubric, reference-side claim splitting and a bidirectional mix. Reference-side is most accurate on ASQA (0.8675-0.975 vs 0.6975-0.85 for holistic) at about 28-29% more tokens. Candidate-side is weakest on completeness labels. Author-reported, on constructed labels.Paper · 2026-03-29GS3LAM: Gaussian Semantic Splatting SLAMGS3LAM fits a Semantic Gaussian Field to RGB-D frames and semantic labels, jointly optimising camera poses and the field with photometric, depth and semantic losses. It adds a scale regulariser (DSR) and random keyframe mapping (RSKM) against forgetting. Author-reported: Replica PSNR 36.26 over eight scenes, mIoU 96.63 over four, 109 FPS rendering.Paper · 2026-03-18How LLMs Distort Our Written LanguageEmpirical study, not a method. Three tests of whether LLMs change the meaning of writing: a 100-person user study, LLM and human edits of 86 pre-LLM essays, and ICLR 2026 peer reviews. Author-reported: heavy LLM use makes essays more neutral and less in the writer's voice; LLM edits shift meaning even under grammar prompts; LLM-labelled reviews cite clarity less and average 4.43 against 4.13.Paper · 2026-03-18How do LLMs Compute Verbal ConfidenceAsks whether LLMs compute a verbal confidence rating only when asked or earlier, during answer generation. Steering, patching, noising, swaps, probes and attention blocking on Gemma 3 27B point to a cache at the post-answer newline that the confidence token reads later. Author-reported: that cache explains variance beyond answer log-probabilities.Paper · 2026-03-11DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot ControlCouples a video diffusion transformer (Cosmos-Predict2.5-2B) with an action diffusion transformer. The action head reads hidden features taken from one video block at a fixed flow timestep, so no frames are generated at control time. Both parts are trained with flow-matching losses. Author-reported, trained from scratch: 98.6% average on LIBERO and 50.8% on 24 RoboCasa-GR1 tasks.Paper · 2026-03-08ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene ReconstructionReconDrive adapts the VGGT 3D foundation model with LoRA and two Gaussian heads to output 4D Gaussian splats from two frames of six-camera driving input in one pass. Vehicles move linearly within a segment. Author-reported on nuScenes: PSNR 32.66 for reconstruction against 29.58 for the best per-scene baseline, at about 15 s per scene.Paper · 2026-03-05cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF RobotsGPU motion stack for high-DoF robots: B-spline trajectories optimised under torque limits (differentiable RNEA), a dense ESDF built from a block-sparse TSDF with PBA+, and LM-then-L-BFGS collision-free IK. Author-reported: 99.7% success under 3 kg payload (cuRobo 77.1%), 99.6% collision-free IK on a 48-DoF humanoid, up to 10x faster ESDF than nvblox.Paper · 2026-03-04MEM: Multi-Scale Embodied Memory for Vision Language Action ModelsMEM gives vision-language-action robot policies two memories: a video encoder that compresses recent camera frames into the ViT without new parameters, and a text memory that a high-level policy rewrites from its previous summary. Built into pi0.6, it runs tasks spanning up to 15 minutes; results are author-reported.Paper · 2026-02-14Nighttime Autonomous Driving Scene Reconstruction with Physically-Based Gaussian SplattingAdds physically based shading to composite 3D Gaussian Splatting driving scenes for night. A global spherical-harmonics module gives diffuse light, per-Gaussian anisotropic spherical Gaussians give specular light, and Reinhard tone mapping runs before rasterization. Author-reported: 31.9 dB PSNR on Waymo and 29.7 dB on nuScenes for reconstruction, above 3DGS, StreetGaussians and OmniRe.Paper · 2026-02-13Doc-to-LoRA: Learning to Instantly Internalize ContextsDoc-to-LoRA (D2L) is a Perceiver hypernetwork that reads a document once through a frozen LLM and writes a rank-8 LoRA adapter, so later queries do not need the document in context. Author-reported: on 2WikiMultihopQA it reaches 0.857 normalized ROUGE-L in 0.21 s per update, versus 0.901 for oracle context distillation at 40 s.Paper · 2026-02-06Optimal Learning Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-DecayDerives the learning-rate schedule that minimises the final-step excess risk of one-pass SGD in power-law kernel regression, via the functional scaling law. Easy tasks (s > 1-1/q) get power decay from step 0; hard tasks hold the stability limit, then decay with exponent 2q-1 (WSD-like). Power decay with p > q-1 reaches the minimax rate without log factors. Small experiments are author-reported.Paper · 2026-02-05DFlash: Block Diffusion for Flash Speculative DecodingDFlash drafts a block of tokens in one parallel pass with a small block-diffusion model. Features from the target LLM are injected as keys and values into every draft layer, and the target verifies the block, so output stays lossless. Author-reported: 4.9x average greedy speedup over autoregressive decoding on Qwen3-4B across seven tasks, up to 6.1x on MATH-500.Paper · 2026-02-02Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal GeometrySplit conformal p-values for each input define a credal set, whose cuts are imprecise highest density regions. Exact O(K) formulas after sorting give generalised Hartley, entropy imprecision, TV-MMI and Tsallis-penalised summaries. Author-reported: with LAC scores these beat fixed-level set size on selective prediction and OOD detection. Regression rank cores cannot separate inputs.Paper · 2026-02-02SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction TuningMixture of LoRA experts for multimodal continual instruction tuning, without replayed data. A router is updated only in directions that matter for the new task, expert updates are scaled by the previous tasks' input covariance, and experts that the current task barely uses are frozen. Author-reported average accuracy: 66.82 on CoIN and 67.12 on UCIT, against 50.58 and 52.06 for MoELoRA.Paper · 2026-01-21HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video UnderstandingTraining-free streaming video method for MLLMs. It splits the KV cache into three memory roles by layer depth: shallow layers favour recency, deep layers attention to frame anchors, middle layers blend both. Deep layers fold evicted tokens into one summary token. Author-reported: Qwen2.5-VL-7B at 4K tokens reaches 79.44% on StreamingBench real-time (base 73.31%).Paper · 2026-01-15DeFlow: Decoupling Behavior-Prior Modeling and Value Maximization for Offline Policy ExtractionDeFlow keeps a multi-step flow-matching policy as a fixed behavior prior and trains a small residual MLP to shift each sampled action toward higher critic value. An adaptive Lagrangian penalty keeps the mean squared shift near a per-task budget, so no gradient passes through the ODE solver. Author-reported: 40 vs 29 (FQL) on OGBench cube-double, 43 vs 30 on puzzle-3x3; SORL leads on navigation.Paper · 2026-01-12Hiking in the Wild: A Scalable Perceptive Parkour Framework for HumanoidsAuthor-reported: a Unitree G1 humanoid runs one end-to-end RL policy from head-camera depth and proprioception to joint targets. A volumetric edge penalty keeps feet off terrain edges, and flat-patch goals replace random velocity commands. Zero-shot hardware runs reach 2.5 m/s over complex terrain.Paper · 2026-01-12Stochastic CHAOS: Why Deterministic Inference Kills, and Distributional Variability Is the Heartbeat of Artifical CognitionArgues that greedy, deterministic LLM inference hides capability, brittleness and tail risk. Compares greedy decoding with T=0.7, top-p 0.9 sampling and majority vote on GLUE paraphrases, few-shot ICL, InstruSum, GSM8K/SVAMP/StrategyQA chain-of-thought and jailbreak safety. Sampling raises author-reported accuracy and robustness; several safety and reasoning tables are simulations.Paper · 2026-01-07Simulated Students in Tutoring Dialogues: Substance or Illusion?Defines turn-level student simulation for tutoring dialogues and scores simulated student turns on seven reference-based metrics. Prompting with GPT models trails SFT and DPO on Llama models. Author-reported: DPO 8B reaches acts 0.684 and cosine 0.739 on Eedi; the Oracle prompt leads only on correctness and errors.Paper · 2026-01-05CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk DecodingAuthor-reported: CycleVLA adds subtask-progress and stop outputs to a VLA. At about 90% progress a VLM decides whether to move on or backtrack, and the retry uses MBR-selected action chunks. On LIBERO-Plus the pi0.5 backbone rises from 65.0 to 86.0 average success. On a real arm the average is 91%.Paper · 2026-01-05Lightweight Transformer Architectures for Edge Devices in Real-Time ApplicationsSurvey of lightweight transformers for edge devices, covering distillation, quantization, pruning, hardware-aware NAS and deployment frameworks. Reports tables of GLUE, SQuAD and ImageNet results and a six-step pipeline; claims 75-96% of full-model accuracy at 4-10x smaller size and 3-9x lower latency. No method or reproducible protocol is given.Paper · 2025-12-30Hardware Acceleration for Neural Networks: A Comprehensive SurveySurvey of hardware acceleration for neural networks: GPUs and tensor cores, TPUs and NPUs, ASIC and FPGA inference engines, LPU-style LLM-serving chips, in-memory, analog and neuromorphic designs. It sorts the field by workload, execution setting and optimisation lever, and gives the serving metrics and KV-cache size formula. It runs no experiments, so it reports no new numbers.Paper · 2025-12-21From Word to World: Can Large Language Models be Implicit Text-based World Models?Treats an LLM as a text-world simulator: given the dialogue history and the action, it predicts the next observation and a success flag. Fine-tuned Qwen2.5-7B reaches 99.87% (ALFWorld) and 98.60% (SciWorld) exact-match next-state accuracy, with consistency ratios of 0.91 to 0.96 on three structured environments. Author-reported agent gains: action checks, synthetic data, RL warm-starts.Paper · 2025-12-21InDRiVE: Reward-Free World-Model Pretraining for Autonomous Driving via Latent DisagreementInDRiVE pretrains a DreamerV3-style world model and explorer in CARLA with no task reward, using ensemble disagreement as the intrinsic reward. The frozen policy is then tested zero-shot on an unseen town and fine-tuned with 10k task-reward steps. Zero-shot success averages 73.2% on the unseen town; collision avoidance after fine-tuning averages 94.8%.Paper · 2025-12-20STORM: Search-Guided Generative World Models for Robotic ManipulationSTORM plans robot manipulation by searching over imagined futures. A diffusion VLA policy proposes candidate actions, a video world model (iVideoGPT, fine-tuned with a reward head) simulates each one, and MCTS with PUCT selects the most-visited root action. Author-reported: 51.0% average success on four SimplerEnv WidowX tasks, against 47.9% for the base CogACT policy.Paper · 2025-12-19OpenAI GPT-5 System CardSystem card for the GPT-5 family: a router picks a fast or a thinking model per query. Reports author-run safety, factuality, deception, health, red-team, chain-of-thought and Preparedness evaluations against o3 and GPT-4o, and treats gpt-5-thinking as High biological and chemical capability with safeguards. Gives no architecture or training recipe; all numbers are author-reported.Paper · 2025-12-19Robotic VLA Benefits from Joint Learning with Motion Image DiffusionAdds a second head to pi-series VLAs: a diffusion transformer that predicts RAFT optical-flow images of the next second, trained jointly with the action head and unused at inference. Author-reported: 97.5% LIBERO average (pi0.5 backbone, 96.9% before) and 58.0% RoboTwin easy-mode average (pi0 45.1%).Paper · 2025-12-17Modeling the Mental World for Embodied AI: A Comprehensive ReviewSurvey of Theory of Mind (ToM) for embodied agents. It separates physical from mental world models with POMDP and predictive-coding formalisms, groups mental-state representations, compares 19 ToM methods in Prompting and Model-Based paradigms, and catalogues 26 benchmarks. No new experiments; numbers are quoted from cited works.Paper · 2025-12-15Learning Terrain Aware Bipedal Locomotion via Reduced Dimensional Perceptual RepresentationsA CNN-VAE compresses 20x40 terrain height maps into a 16-value latent, which a hierarchical PPO policy uses with a reduced-order robot state to command task-space feet and speed on the Digit robot. Author-reported: over 80% success on all tested terrains in simulation, and at least 2x fewer training samples than end-to-end baselines.Paper · 2025-12-15OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy LearningAugE-Toolkit cross-paints existing robot demonstrations onto other arms: it segments the source arm, inpaints the background and replays the trajectory in simulation. Applied to 16 OXE datasets with 9 robot embodiments, giving over 4.4M trajectories. Author-reported: OpenVLA-OFT and pi0 fine-tuned on it gain 24% and 45% on average in real tests on unseen robot-gripper pairs.Paper · 2025-12-13Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use ScaleDevelops a 13-item self-report scale of critical thinking in AI use (verification, motivation, reflection) over six studies (N = 1341), with a higher-order CFA, sex invariance, test-retest reliability and links to fact-checking behaviour. All results are author-reported.Paper · 2025-12-10Interpretable Embeddings with Sparse Autoencoders: A Data Analysis ToolkitUses a sparse autoencoder trained on Llama 3.3 70B activations to turn each document into an interpretable embedding of concept latents (max-pooled over tokens). The embedding supports dataset diffing, correlation discovery, controllable clustering and property-based retrieval. Author-reported: 2-8x lower cost than LLM baselines; retrieval close to dense embeddings.Paper · 2025-12-09Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language NavigationDualVLN splits vision-and-language navigation into a 7B VLM (System 2) that predicts a pixel waypoint and a latent goal, and a 384-wide diffusion transformer (System 1) that turns them into 32 waypoints. The two run asynchronously. Author-reported: 64.3 SR on R2R-CE val-unseen with RGB only (StreamVLN 56.9), and 51.60 SR on VLN-PE val-unseen without training on VLN-PE.Paper · 2025-12-08Unified Camera Positional Encoding for Controlled Video GenerationUCPE encodes each video token's viewing ray in a local ray frame, so lens distortion and FoV enter attention, and adds a gravity-aligned latitude-up map for absolute pitch and roll. A parallel attention adapter (35.6M trainable parameters on Wan2.1 T2V 1.3B) injects it. Author-reported: lower rotation and translation errors than ReCamMaster and Wan CameraCtrl on a synthetic 48k-clip UCM dataset.Paper · 2025-12-07A Novel Deep Neural Network Architecture for Real-Time Water Demand ForecastingShort-term water demand forecasting with a small GRU network (DCGRU) whose inputs add k-means class features (m=4), plus an EDCGRU variant that inserts one virtual value between readings to reduce error at extreme points. Author-reported one-step MAPE 1.26-1.31% on two district metering areas, with AIC about six times lower than a GRUN baseline.Paper · 2025-12-07VideoVLA: Video Generators Can Be Generalizable Robot ManipulatorsVideoVLA starts from the pretrained CogVideoX-5B video diffusion transformer and denoises future video latents and a 6-step, 7-D action chunk jointly, conditioned on an instruction and the current image. Author-reported SIMPLER average over 12 tasks: 63.0 vs 62.6 for CogACT. Novel-object and new-skill generalisation: 65.2 and 48.6 vs 42.4 and 20.4.Paper · 2025-12-05Training-Time Action Conditioning for Efficient Real-Time ChunkingTraining-time RTC: instead of inpainting the action prefix at inference, simulate inference delay during training, feed the prefix as clean actions with flow timestep 1, and compute the loss only on the postfix. No inference overhead. Author-reported: beats inference-time RTC in simulation at delays of 2 and above; matches its success and speed on real box building and espresso with pi0.6.Paper · 2025-12-04Learning to Orchestrate Agents in Natural Language with the ConductorTrains a 7B Conductor LLM with GRPO to write a 1-to-5-step workflow for a pool of worker LLMs: model ids, natural-language subtasks and per-step access lists. Author-reported: in unconstrained runs it beats every listed worker on all seven benchmarks, e.g. LiveCodeBench v6 83.93 vs 82.90 for GPT 5.Paper · 2025-12-04SIMA 2: A Generalist Embodied Agent for Virtual WorldsSIMA 2 is a Gemini Flash-Lite agent that plays 3D games from screen pixels and keyboard-and-mouse actions. It is fine-tuned on human gameplay and Gemini-written reasoning and dialogue, then trained with RL on verifiable tasks. Author-reported: about twice SIMA 1's success in training games, over 10% gains on two held-out games, and self-improvement scored by a Gemini reward model.Paper · 2025-12-04The Universal Weight Subspace HypothesisAuthor-reported: across hundreds of fine-tuned networks (LoRAs, ViTs, LLaMA-3-8B, GPT-2, Flan-T5, ResNet-50), each layer's weights share a few principal directions that explain most variance. Projecting new models onto them loses little (ViT: 94.1 vs 94.4 IID, 87.8 vs 91.3 OOD) and cuts storage by about 19x (LoRAs) to 100x (ViTs).Paper · 2025-12-03RoboScape-R: Unified Reward-Observation World Models for Generalizable Robotics Training via RLTrains robot policies inside a learned world model. An action model predicts the next frame from the policy's actions, and a text model imagines a goal frame from the instruction. The reward combines an LPIPS term to that goal with a sparse done-signal term. Author-reported: OpenVLA trained this way reaches 84.1% on unseen pick-and-place scenes, against 54.5% for RL with a simulator reward.Paper · 2025-12-02CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement LearningCUDA-L2 trains an LLM with staged reinforcement learning, using measured kernel speed minus numerical error as reward, to write FP16 HGEMM CUDA kernels for 1,000 (M, N, K) shapes on A100. Author-reported offline mean: +11.4% over cuBLASLt-AutoTuning and +22.0% over torch.matmul.Paper · 2025-12-02Vehicle Dynamics Embedded World Models for Autonomous DrivingVDD splits a Dreamer-style world model into an ego-vehicle dynamics model with physical parameters and an environment model driven by ego states. A learned goal policy and a PID controller act. PAD (adjust actions at deployment) and PAT (imagine shifted vehicle dynamics in training) target changed mass and steering limits. Author-reported MetaDrive roundabout success: 0.70 vs 0.53 for DreamerV3.Paper · 2025-12-01AlignSAE: Concept-Aligned Sparse AutoencodersTrains a sparse autoencoder on one GPT-2 layer in two stages: reconstruction first, then supervised post-training that binds each ontology relation to its own latent slot and decorrelates it from free features. Author-reported at layer 6: diagonal binding 1.00 on training templates, 0.912 on unseen; 0.847 swap success at alpha=2; 2-hop swap 7.86% vs 1.5-2.7% for a standard SAE.Paper · 2025-12-01Learning Sim-to-Real Humanoid Locomotion in 15 MinutesA recipe for sim-to-real humanoid RL built on off-policy FastSAC and FastTD3 with massively parallel simulation: joint-limit action bounds, layer norm, averaged critics, tuned exploration and a minimal reward. Author-reported: locomotion policies for Unitree G1 and Booster T1 trained in 15 minutes on one RTX 4090.Paper · 2025-11-28Hunyuan-GameCraft-2: Instruction-following Interactive Game World ModelHunyuan-GameCraft-2 is a 14B image-to-video mixture-of-experts model that turns free-form text, keyboard and mouse inputs into interactive game video. It is distilled into a causal, autoregressive generator with a sink-token KV cache, trained with randomized long-video tuning, and reaches 16 FPS. Author-reported: top InterBench scores against three video baselines.Paper · 2025-11-24Mixture of Horizons in Action ChunkingMixture of Horizons (MoH) trains a vision-language-action policy's action head on several action-chunk lengths at once. The lengths share one transformer pass and a learned per-step softmax gate fuses them, with a balance loss. A cross-horizon consensus rule can cut execution length at inference. Reports 99% average on LIBERO for pi0.5 with MoH (97.7% baseline). Author-reported.Paper · 2025-11-16Fine-Grained Representation for Lane Topology ReasoningTopoFG describes each lane with 11 ordered fine-grained queries instead of one, seeded by a BEV mask prior and point-order priors. Only the first and last query of each lane feed a pairwise connectivity head, trained with fixed-target denoising groups. On OpenLane-V2 subset_A it reports OLS 48.0 against 44.1 for TopoLogic (author-reported).Paper · 2025-10-30A Sliding-Window Filter for Online Continuous-Time Continuum Robot State EstimationEstimates the pose, velocity and strain of a continuum robot along its length and over time by solving a factor-graph MAP problem on a short sliding window. The oldest time step is marginalised into a prior with the Schur complement. Author-reported: tip RMSE within 0.01 percentage points of the batch smoother on four of five trajectories, and 8.7 to 10.2 ms per step for a 0.1 s window.Paper · 2025-10-21EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUsMeasures latency, power and energy of distilled DeepSeek-R1 models (1.5B, 8B, 14B) on a Jetson AGX Orin with vLLM. Fits analytic latency and energy models (total latency MAPE about 0.5%), then compares token budgets, prompt-based length control, the L1 budget-aware model, parallel sampling and AWQ W4 for accuracy, latency and cost.Paper · 2025-10-20LLM-as-a-Prophet: Understanding Predictive Intelligence with Prophet ArenaProphet Arena is a live benchmark that collects prediction-market events, gives every LLM the same news and market context, and scores probabilistic forecasts by Brier score, calibration error and simulated market return. Frontier models match the market baseline on Brier score, but none breaks even on return (author-reported).Paper · 2025-10-19Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World ModelsIR-WM is a vision-centric driving world model that predicts only the residual change of a bird's-eye-view scene state, not the whole future scene, and aligns features to limit error build-up. It forecasts 4D occupancy and plans trajectories. Author-reported on nuScenes: average planning L2 0.53 m vs 0.85 m for Drive-OccWorld.Paper · 2025-10-17Real-Time QP Solvers: A Concise Review and Practical Guide Towards Legged RobotsReview and selection guide for real-time convex QP solvers in legged-robot MPC and whole-body control. Compares four solver families (active-set, interior-point, operator-splitting, augmented Lagrangian) and code generation, mapped to problem structure. No new experiments; benchmark results are relayed. Guidance: structured IPM (HPIPM) for long-horizon MPC, dense active-set or ProxQP for WBC.Paper · 2025-10-14Hybrid Gaussian Splatting for Novel Urban View SynthesisChallenge entry for novel street-view synthesis: a 3D Gaussian Splatting model, with COLMAP-densified points, renders target views and a single-step diffusion enhancer, fine-tuned on 36k curated pairs, removes artifacts. Author-reported 0.432 on the leaderboard, second of 28.Paper · 2025-10-06Think Then Embed: Generative Context Improves Multimodal EmbeddingThink-Then-Embed (TTE) has a multimodal LLM write a short reasoning text about a query or target, then embeds the input together with that text. A 72B teacher writes the reasoning and a Qwen2-VL 7B embedder reaches 71.5% overall on MMEB-V2 (author-reported, 78 tasks). A 7B reasoner finetuned on teacher traces reaches 68.6%.Paper · 2025-10-02ExGRPO: Learning to Reason from ExperienceAdds experience replay to RLVR for reasoning LLMs: correct rollouts are stored by success count, fully solved questions are retired, and half of each batch replays the lowest-entropy stored answer with a damped importance ratio. Author-reported: 48.3 vs 45.3 average on six math benchmarks (Qwen2.5-Math-7B, on-policy RLVR baseline).Paper · 2025-09-29DelRec: learning delays in recurrent spiking neural networksLearns a real-valued delay per recurrent neuron (axonal) or connection (synaptic) in a spiking RNN, trained with surrogate gradients. Each spike is spread over neighbouring time steps by a triangular kernel whose width is annealed towards zero; delays are rounded to integers afterwards. Author-reported: 82.42% on Spiking Speech Commands and 96.63% on PS-MNIST with plain LIF neurons.Paper · 2025-09-29Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view VideosFeed-forward model that turns uncalibrated 4-view human videos into dynamic 3D Gaussians for novel-view and novel-time rendering. A VGGT backbone with a temporal state token predicts per-frame Gaussians; a motion head predicts dense 3D motion, trained with retargeting and flow losses. Author-reported on DNA-Rendering: novel view and time PSNR 29.04, against 18.03 for L4GM.Paper · 2025-09-29Watermarking Diffusion Language ModelsAdapts red-green watermarks to diffusion language models, which fill tokens in any order. Each diffusion step tilts the logits by the gradient of an expected green-token count, taken over the distribution of not-yet-decoded context hashes. Detection is the unchanged binomial test. Author-reported: 99% TPR at 1% FPR on LLaDA and Dream with a small quality cost.Paper · 2025-09-26DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for Autonomous Travel Planning AgentsDeepTravel trains a travel-planning LLM agent end to end with RL. A cached tool sandbox gives stable tool responses, a trajectory-level and a turn-level verifier give a binary reward, and failed queries are replayed from a buffer. The 32B model reports an 82.58% online final pass rate over a three-month deployment (author-reported).Paper · 2025-09-24Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained ModelsTargeted attack on multimodal embedding models. The image adversarial example is aligned to several text proxies of the target concept and to benign source-image proxies, so it generalises to unseen wordings and evades four anomaly detectors. Author-reported: 98.31% classification ASR and 77.72% retrieval R@1 ASR without defense; 84.17% and 61.80% under the anomaly-detection defense.Paper · 2025-09-24VisualMimic: Visual Humanoid Loco-Manipulation via Motion Tracking and GenerationTwo-level sim-to-real framework for humanoid loco-manipulation. A task-agnostic keypoint tracker, distilled from a motion tracker trained on human motion, follows root, head, hand and foot commands. A depth-based task generator, distilled from a state-based teacher, produces them. Noise and human-motion action clipping stabilise training. Author-reported zero-shot transfer to a Unitree G1.Paper · 2025-09-21FILIC: Dual-Loop Force-Guided Imitation Learning with Impedance Torque Control for Contact-Rich Manipulation TasksFILIC pairs a Transformer imitation policy (RGB plus estimated end-effector force, 25 Hz) with a joint-space impedance torque loop (2 kHz). The force comes from joint torques through a Jacobian pseudo-inverse and a MuJoCo digital twin, with no force/torque sensor. Author-reported: simulated peg insertion 68.0% to 90.0%; real tasks 26.6-46.7% (position only) vs 76.6-83.3% (estimated force).Paper · 2025-09-18Stochastic Bilevel Optimization with Heavy-Tailed NoiseNested-loop normalized stochastic bilevel method (N2SBA) for smooth bilevel problems with a strongly convex lower level and heavy-tailed gradient noise: clipped SGD in the inner loops, normalized SGD in the outer loop. Its SFO complexity is O~(kappa^((7p-3)/(p-1)) sigma^(p/(p-1)) eps^(-(4p-2)/(p-1))); at p=2 it matches the best bounded-variance first-order bound. Also N2SGDA for minimax.Paper · 2025-09-04FPC-VLA: A Vision-Language-Action Framework with a Supervisor for Failure Prediction and CorrectionA VLA policy (vision encoders, Llama2, diffusion action head) plus a Qwen2.5-VL supervisor. The supervisor is queried only when the gripper state changes; if it answers "No", it adds a directional correction. A cosine- and time-weighted fusion averages past action predictions. Author-reported: 86.9% LIBERO average and 64.6% task success on SIMPLER WidowX.Paper · 2025-08-21DriveSplat: Unified Neural Gaussian Reconstruction for Dynamic Driving ScenesDriveSplat models a driving scene as neural Gaussians: a hashed multi-scale background with near, mid and far detail levels, and tracked actors with a rigid pose plus anchor-level MLP deformation. Depth and normal priors supervise training. Author-reported: best novel-view PSNR on Waymo (33.83) and KITTI (24.53).Paper · 2025-08-16LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with MambaLocoMamba trains a quadruped locomotion policy end to end with PPO. Proprioceptive and depth-image tokens are fused by two stacked selective state-space (Mamba) layers instead of self-attention. Author-reported in PyBullet: on the training terrain, 48.9% higher return than a Transformer fusion baseline, 48.9% fewer collisions and 30.4% longer distance, with gains on unseen terrain.Paper · 2025-08-10Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language ModelsA parallel safety benchmark: 972 harmful seed prompts rendered into 24 text, image, video and audio combinations (23,328 cases). Five LLM judges label comprehension, harm and refusal; scores count only understood answers and give a Safety-score and a cross-modal consistency score (CMSC). Author-reported: of 11 models, only 3 exceed 0.6 on both average Safety-score and CMSC.Paper · 2025-08-07Information-Theoretic Graph Fusion with Vision-Language-Action Model for Policy Reasoning and Dual Robotic ControlGF-VLA builds scene graphs of hand-object and object-object interactions from RGB-D human demonstrations, using entropy and mutual information over object positions. A 7B LLaMA 2 model with separate LoRA heads plans with chain-of-thought and outputs Cartesian commands for a dual-arm robot. Author- reported: 94% grasp success, 89% placement accuracy and 90% task success.Paper · 2025-08-06There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement LearningModels multi-turn LLM red teaming as an MDP solved by two-level PPO: a high-level policy picks an attack guide per turn, a low-level policy writes the utterance token by token. A pairwise token-masking marginal reward assigns credit. Author-reported: 75.2% HarmBench ASR@5 against Llama-3.1-8B-Instruct, above the baselines the authors ran.Paper · 2025-08-05Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step EntropyScores each step of a long chain-of-thought by the mean entropy of its tokens and replaces the lowest-entropy 80% of steps with [SKIP] markers. Static pruning cuts thinking tokens by 27-45% on Math500 and AIME with small accuracy changes (DeepSeek-R1 7B and 14B, Qwen3-8B). SFT then GRPO teaches a model to emit [SKIP] itself, using 35-57% fewer tokens. Author-reported.Paper · 2025-08-04AI Scientists for Building Virtual-Cell ModelsCellForge is a team of LLM agents that reads a single-cell perturbation dataset, surveys literature, debates model designs in a critic-moderated panel, then writes, trains and validates a perturbation predictor with no human-specified architecture. Author-reported: PCC 0.9883 on Adamson CRISPRi (CPA-X) and MSE_DE 0.1736 on Norman CRISPRa (scGen-X).Paper · 2025-07-31iLRM: An Iterative Large 3D Reconstruction ModeliLRM predicts 3D Gaussians from posed images with a transformer. Viewpoint tokens built from Plücker rays are refined over 12 layers by per-view cross-attention to image tokens, then mixed by self-attention across views. Author-reported: 28.65 PSNR on RE10K at 2 views, 256x256, 0.025 s; 0.53 s for 32 views at 540x960.Paper · 2025-07-29Evaluation and Benchmarking of LLM Agents: A SurveySurvey of how LLM agents are evaluated. Proposes a two-dimensional taxonomy of evaluation objectives (behavior, capabilities, reliability, safety) and process (interaction mode, data, metrics, tooling, context). Adds enterprise challenges and future directions. No experiments and no headline numbers.Paper · 2025-07-20Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile GripperHandheld gripper with flexible tactile pads, and a visuo-tactile encoder (CLIP ViT-B/16, tactile CNN, two cross-attention steps) pretrained by masked tactile reconstruction on over 2,700 in-the-wild demos, then used in a diffusion policy. Author-reported: test-tube whole-task success 0.85 vs 0.70 without pretraining and 0.25 vision-only (20 trials).Paper · 2025-07-18DREAMS: Density Functional Theory Based Research Engine for Agentic Materials SimulationDREAMS is a hierarchical multi-agent LLM framework for periodic DFT (Quantum ESPRESSO). A shared canvas holds data, and a multi-tier guard checks each parameter deterministically and with LLM judges, tracing values to their sources. On Sol27LC, MAPE is 0.36-1.00% vs expert DFT; on CO/Pt(111), the FCC-ontop energy difference is within 2 meV of the expert. The guard raises input tokens about 13x.Paper · 2025-07-10DrugMCTS: a drug repurposing framework combining multi-agent, RAG and Monte Carlo Tree SearchDrugMCTS runs a Qwen2.5-7B-Instruct model, without fine-tuning, through five retrieval and analysis agents inside Monte Carlo tree search. Paths are scored by self-consistency plus a yes/no interaction vote. Author-reported DrugBank and KIBA recall is 0.4466 and 0.4224 at selection = ground-truth count, and 0.5534 and 0.4924 at GT+3, beating GPT-4o-mini, DeepSeek-R1 and a deep-learning ensemble.Paper · 2025-07-08OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent SafetyDocker benchmark (MIT code) where LLM agents use real shell, file, browser, IPython and chat tools on about 350 multi-turn tasks across eight risk categories, scored by rules on the final state plus a GPT-4.1 judge. Author-reported: unsafe rates on safety-vulnerable tasks range from 49.06% (Claude Sonnet 4) to 72.73% (o3-mini) across seven models.Paper · 2025-07-07wd1: Weighted Policy Optimization for Reasoning in Diffusion Language ModelsRatio-free RL for masked diffusion LLMs: group-relative advantages become softmax weights on a weighted log-likelihood with a negative-sample term, needing one likelihood approximation. On LLaDA-8B-Instruct, author-reported: Sudoku 76.4 vs 17.6 for reproduced d1 (256 tokens), GSM8K 82.3 (512 tokens); wd1++ reaches 44.2 on MATH500.Paper · 2025-07-05Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain HyperelasticityP-DivGNN is a graph neural network that predicts the stress at every node of a periodic micro-structure mesh from the cell's mean stress. It adds periodic edges and a penalty on the discrete stress divergence. Author-reported: NMSE on linear elastic data falls from 1.13e-2 (periodic GNN) to 1.08e-2; on hyperelastic data inference is about 500x faster than FE.Paper · 2025-06-25TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and AnalysisTESSERA is a pixel-wise self-supervised model over Sentinel-1/2 time series. It learns a 128-d embedding per 10 m pixel per year with Barlow Twins, random sparse date sampling, global shuffling and mixup regularisation. Author-reported: small heads on its embeddings often beat other foundation-model embeddings on six benchmarks, most clearly with few labels.Paper · 2025-06-22RoboArena: Distributed Real-World Evaluation of Generalist Robot PoliciesRoboArena ranks generalist robot policies from crowd-sourced, double-blind A/B real-robot trials run by evaluators at seven institutions on DROID. A Bradley-Terry model with latent task buckets, fit by EM, corrects for varying tasks. Reported: 612 pairwise comparisons; under simulated drift Pearson r 0.838 vs 0.692 for a standardized protocol. Author-reported.Paper · 2025-06-20AQUA20: A Benchmark Dataset for Underwater Species Classification under Challenging ConditionsIntroduces AQUA20, 8,171 underwater images in 20 marine classes with a fixed 6,559/1,612 train/test split, and fine-tunes 13 ImageNet-pretrained CNNs and transformers on it. Best is ConvNeXt: 90.69% top-1, 98.82% top-3, 88.92% macro-F1, author-reported.Paper · 2025-06-11Understanding and Mitigating Numerical Sources of Nondeterminism in LLM InferenceSame model, same prompt, same seed: under BF16, changing GPU count, GPU type or batch size changes greedy outputs, because floating-point addition is non-associative. FP32 removes most of it. LayerCast stores weights in 16-bit and casts each to FP32 just before its matmul. Author-reported: BF16 greedy AIME'24 accuracy std 9.15% vs 0 for FP32 (DeepSeek-R1-Distill-Qwen-7B); LayerCast tracks FP32.Paper · 2025-06-09Primal-Dual iLQR for GPU-Accelerated Learning and Control in Legged RobotsA GPU-parallel SQP-style MPC for legged robots, in JAX. It solves the primal-dual KKT system with a parallel associative scan of LQR value functions and a 10-step parallel filter line search. Author-reported, varying horizon: up to 60% lower runtime than crocoddyl (whole-body) and up to 700% vs acados (single-rigid-body). 16 robots in one solve, under 25 ms.Paper · 2025-06-09The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification TasksBenchmarks 12 feature-scaling methods plus no scaling on 14 algorithms and 16 UCI tabular datasets (9 classification, 7 regression). Each scaler is fitted on the training split only. Author-reported: ensembles (RF, LightGBM, CatBoost, XGBoost) are largely insensitive to the scaler, while LR, SVMs, TabNet and MLPs are not. Code is MIT-licensed.Paper · 2025-06-07The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem ComplexityCompares reasoning (thinking) LLMs with their non-thinking twins on four controllable puzzles (Tower of Hanoi, Checker Jumping, River Crossing, Blocks World) at matched token budgets. Finds three complexity regimes, accuracy collapse to near zero beyond a model-specific size, and falling thinking-token use near collapse. Results are author-reported.Paper · 2025-06-05GEX: Democratizing Dexterity with Fully-Actuated Dexterous Hand and Exoskeleton GloveGEX is a low-cost teleoperation kit: a three-finger hand (GX11, 11 DoF) and a matching exoskeleton glove (EX12, 12 DoF), every joint motor-driven, 3D-printed ABS parts, about $600 per device. Glove fingertip keypoints are retargeted to hand joints by optimization. Author-reported: one real cup grasp and three simulated tasks, shown qualitatively, with no success rates or baselines.Paper · 2025-06-04TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent SystemsReview of trust, risk and security management (TRiSM) for LLM-based multi-agent systems. It adapts five pillars (explainability, ModelOps, security, privacy, lifecycle governance), proposes a four-class risk taxonomy, and defines two metrics: Component Synergy Score and Tool Utilization Efficacy. No experiments, so no headline numbers.Paper · 2025-06-02Small Language Models are the Future of Agentic AIPosition paper: small language models (SLMs) are powerful enough, more suitable and more economical than LLMs for the narrow, repetitive calls made by agents, so they should be the default there. Gives a five-step LLM-to-SLM conversion procedure. No experiments of its own; cited results and author case-study estimates are unverified.Paper · 2025-05-23MMMG: A Comprehensive and Reliable Benchmark for Multitask Multimodal GenerationMMMG is a benchmark of 55 tasks and 1288 instructions for image, audio, interleaved image-text and interleaved audio-text generation. Each task is verifiable or near-verifiable: a program, a VLM or audio model judge with rubrics, or reference-based similarity. Reported average human agreement is 94.4%. GPT Image reaches 70.7% on image generation (author-reported).Paper · 2025-05-23TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous DrivingTopoPoint detects lane endpoints as separate queries and lets them exchange features with lane queries through merged self-attention and a point-lane graph network. At inference, detected endpoints pull nearby lane ends to a shared point. On OpenLane-V2 subset_A it reports OLS 48.8 and DET_p 52.6 (author-reported).Paper · 2025-05-21SAKE: Structured Agentic Knowledge Extrapolation for Complex LLM Reasoning via Reinforcement LearningSAKE trains Qwen2.5 models with GRPO to call two knowledge-graph tools in a three-turn rollout: extract entities, keep relevant concept groups, then reason over retrieved triplets by analogy. Reward: format plus exact-match answer. Author-reported: Qwen2.5-7B reaches 75.4% weighted average on UMLS QA (GPT-3.5-Turbo with GIVE: 70.1%) and 81.3% on CommonsenseQA, with over 90% fewer tokens.Paper · 2025-05-20DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning ModelsDRP has a teacher LLM split a student model's long chain-of-thought into skill-labelled steps, prune them (keep, delete, compress, merge), then fine-tune the student on the pruned traces with LoRA. Author-reported: R1-Distill-Qwen-7B on GSM8K goes from 917 to 328 tokens while accuracy rises from 91.7% to 94.1%.Paper · 2025-05-20This Time is Different: An Observability Perspective on Time Series Foundation ModelsToto is a 151M-parameter decoder-only forecaster for observability metrics. It scales each patch with past data only, alternates time-wise and variate-wise attention in an 11:1 ratio, and outputs a Student-T mixture trained with a robust loss. Author-reported: lowest MASE and CRPS on the new BOOM benchmark (0.617 and 0.375) and the best GIFT-Eval average rank (5.495).Paper · 2025-05-17Solver-Informed RL: Grounding Large Language Models for Authentic Optimization ModelingSIRL fine-tunes Qwen2.5 7B and 32B instruct models with RL. The reward comes from running generated optimization code in a solver: format, execution and objective value. A partial KL penalty covers only the model and code sections. Author-reported macro accuracy: 60.6 (7B) and 68.3 (32B) over five benchmark columns.Paper · 2025-05-15AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and ChallengesNarrative review that separates single-entity AI Agents (an LLM plus tools, one bounded task) from multi-agent Agentic AI (specialized agents, shared memory, orchestration). Compares both with Generative AI across five dimensions, maps application areas, lists challenges and ten mitigation mechanisms. It reports no experiments; every comparison is a qualitative author judgement.Paper · 2025-05-13Dataflow & Tiling Strategies in Edge-AI FPGA Accelerators: A Comprehensive Literature ReviewSurvey of FPGA dataflow and tiling for edge DNN accelerators. It compares four dataflow styles (weight-, output-, row-stationary, no-local-reuse), the MAESTRO and Timeloop analytical tools, multi-level tiling and HLS/DSL/RTL design flows, then summarises FINN, FINN-R, FlightLLM and SSR in one table. It runs no new experiments; numbers are quoted from cited works.Paper · 2025-05-06RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement LearningExtends R1-Zero-style GRPO reinforcement learning into a closed loop with a simulated racing car. Small Qwen2.5 LLMs (1.5B, 3B) learn to tune the car's MPC parameters from simulator rollouts, after SFT. Qwen2.5-3B reaches a 63.3% control adaptability score against 58.5% for GPT-4o on one evaluation track. The paper states a 20.2-point gain for 1.5B, but Table 2 implies 30.2.Paper · 2025-05-03BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning ModelsBOOM is a benchmark for predicting molecular properties out of distribution. For 10 properties, the molecules with the lowest kernel-density values form the OOD test set. Of 15 models tested, none is strong on all tasks: the top model's average OOD error is about 3x its in-distribution error. Pretraining chemical language models improves in-distribution error but not OOD error.Paper · 2025-04-28From LLM Reasoning to Autonomous AI Agents: A Comprehensive ReviewSurvey of LLM reasoning and autonomous AI agents, 2019 to 2025. Groups about 60 benchmarks into a taxonomy, reviews agent frameworks, applications and training datasets, and compares the MCP, ACP and A2A protocols. Runs no experiments of its own; its figures are author-reported in the works it cites.Paper · 2025-04-25Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providersMaps clinical therapy guidance into features of good therapy, then tests gpt-4o and Llama 3.1 for stigma in vignettes and for replies to crisis stimuli (delusions, suicidal ideation, mania, hallucinations, OCD). Author-reported: 38% (gpt-4o) to 75% (Llama 3.1 405B) stigmatizing answers overall; concludes LLMs should not replace therapists.Paper · 2025-04-24Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMsUniME turns a multimodal LLM (Phi-3.5-V, LLaVA-1.6) into a universal image-text embedder in two stages: text-only distillation of NV-Embed V2's batch similarity structure into the LLM, then InfoNCE tuning with false-negative filtering and k=8 sampled hard negatives. Author-reported: 70.7 overall Precision@1 on MMEB (36 datasets) with LLaVA-OneVision-7B, against 63.3 for VLM2Vec (LLaVA-1.6).Paper · 2025-04-13Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models?Open-weight LLMs (Llama 3.1/3.2, Qwen 2.5) propose stage-based CNNs for microcontrollers. Candidates are checked for MACs and SRAM, mini-trained, and kept on a Pareto front of accuracy, MACs and parameters. The best are fully trained and distilled from a ViT-B/16 teacher. Author-reported CIFAR-100 accuracy: 74.50% (LMaNet-Elite) on an STM32H743.Paper · 2025-04-12Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a ReviewSemi-systematic review of explainable AI (SHAP, LIME, Grad-CAM, PDP, LRP) in food quality. Proposes a five-topic food-quality taxonomy and sorts studies by data type and explanation type. Pictorial data and visual explanations dominate; spectral and time-series data are rarely explained. Survey of over 200 works (author-reported count), not a new method.Paper · 2025-04-10Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible CorporaFindings of the first BabyLM Challenge: 31 teams pretrained English LMs on at most 100M words (10M in Strict-Small), scored on grammar, GLUE, MSGS and a hidden supplement. LTG-BERT-based ELC-BERT leads the Strict and Strict-Small tracks (aggregate 0.74 and 0.66). Most curriculum-learning attempts did not help. Author-reported.Paper · 2025-04-09VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-TuningVideoChat-R1 fine-tunes Qwen2.5-VL-7B with GRPO and rule-based rewards (IoU for time ranges and boxes, answer accuracy, output format) on about 18k short-video samples. Author-reported: Charades-STA mIoU rises from 29.0 to 60.8 and GoT-10k tracking overlap from 12.6 to 43.8 over the base model.Paper · 2025-03-31A Concise Survey on Lane Topology Reasoning for HD MappingSurvey of lane topology reasoning for HD maps. Sorts methods into procedural modeling, aerial-imagery and onboard-sensor families, defines the APLS, TLTS, DET and TOP metrics, and tabulates the scores reported for them on Topo-Boundary and OpenLane-V2. It proposes no new method and has no code.Paper · 2025-03-31HACTS: a Human-As-Copilot Teleoperation System for Robot LearningHACTS is a low-cost leader-follower teleoperation arm whose joints are synchronised both ways, so a human can take over a running policy and their corrections are recorded. Author-reported: with correction data, ACT success on three real tasks rose on the failure-correction setting; RLPD-HACTS went from 50% to 80% on CloseBin after 45 min of online training.Paper · 2025-03-30JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior SynchronizationJoint audio-video diffusion transformer: one text prompt yields a video and its sound together. A contrastively trained estimator turns the prompt into spatial and temporal prior tokens that condition both branches. Author-reported JavisScore 0.154 on JavisBench, vs 0.151 for the best cascaded baseline.Paper · 2025-03-29Agentic Large Language Models, a surveySurvey of agentic LLMs, language models that reason, act and interact. It sorts the literature into reasoning, acting and interacting, summarised in three tables of 102 rows, and proposes a research agenda on training data, hallucination, scale, self-reflection and safety. It reports no experiments of its own; the claims are the authors' summaries of cited work.Paper · 2025-03-20Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tFine-tunes the 1.5B DeepSeek-R1-Distill-Qwen model with GRPO and rule-based rewards (answer correctness, format, cosine length scaling) on 7k curated math problems in 24 h on four A40 GPUs. Author-reported: AIME24 46.7% and about \$42 training cost. Gains appear early, then training becomes unstable past about 200 steps.Paper · 2025-03-18GR00T N1: An Open Foundation Model for Generalist Humanoid RobotsGR00T N1 is an open vision-language-action model for humanoid robots. A Eagle-2 VLM reads images and language, and a diffusion transformer trained with flow matching outputs 16-step action chunks. It is pretrained on real, simulated, neural-generated and human-video data with learned latent actions. Author-reported: 45.0% average simulation success vs 33.4% for Diffusion Policy.Paper · 2025-03-13SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video AnalysisZero-shot multi-agent workflow for five robotic-surgery video QA tasks. An orchestrator routes each multiple-choice question to a visual or a knowledge stream; agents write task-specific chain-of-thought answers, a panel checks action answers against a knowledge graph, and retrieval supports knowledge questions. Author-reported: 61.73% overall with GPT-4o vs 31.44% for GPT-4o alone.Paper · 2025-03-06Temporal Analysis of NetFlow Datasets for Network Intrusion Detection SystemsRebuilds four NIDS benchmarks (UNSW-NB15, BoT-IoT, ToN-IoT, CSE-CIC-IDS2018) as NetFlow records with nProbe, adding flow start and end times and per-flow inter-packet arrival time statistics. Analyses them with flow-length histograms, per-minute series and spectrograms. Headline: four labelled datasets of 2.4M to 27.5M flows. No model is trained; attack-pattern claims are visual only.Paper · 2025-03-05OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature ExtractionOTTER keeps CLIP frozen. Per camera, it weights CLIP's last-layer patch features by their similarity to each instruction token and pools them into a causal transformer that predicts 10-D delta end-effector action chunks. Author-reported: 62% on 70 unseen real-robot pick-and-place trials, versus 12% for fine-tuned Octo and 9% for fine-tuned OpenVLA.Paper · 2025-03-03Visual-RFT: Visual Reinforcement Fine-TuningVisual-RFT fine-tunes Qwen2-VL vision-language models with GRPO and rule-based rewards (IoU and confidence for boxes, exact class match for classification, plus a format reward) instead of imitating labelled answers. Author-reported: one-shot fine-grained classification averages 80.3 vs 51.7 for SFT and 56.0 for the base model; Qwen2-VL-2B gains 21.9 mAP over base on 2-shot COCO detection.Paper · 2025-02-28Emergent Abilities in Large Language Models: A SurveyA survey, not a method. It reviews what "emergent abilities" of LLMs means, whether sudden capability jumps survive metric changes, and what mechanisms, in-context learning, reasoning models, agents and harmful behaviours go with them. Key claim: an abrupt curve alone does not show emergence; a checkable internal mechanism is needed.Paper · 2025-02-27Fine-Tuning Vision-Language-Action Models: Optimizing Speed and SuccessStudies how to fine-tune the OpenVLA vision-language-action model and proposes OpenVLA-OFT: parallel decoding with 8-step action chunks, continuous actions and an L1 regression head. Reports 97.1% average success on LIBERO (up from 76.5%) and 26x action throughput. On a bimanual ALOHA robot, the FiLM-augmented OFT+ variant outperforms pi0 and RDT-1B on its tasks.Paper · 2025-02-26Accelerating scientific discovery with Co-ScientistCo-Scientist is a Gemini 2.0 multi-agent system that generates, critiques, ranks (Elo tournament) and evolves research hypotheses, scaling test-time compute. Author-reported: top-hypothesis Elo rises over time, experts prefer its output on 11 goals, and lab assays validate drug repurposing candidates for AML. No public code.Paper · 2025-02-26Model Adaptation: Unsupervised Domain Adaptation without Source DataAdapts a source-trained image classifier to unlabeled target data without source data. 3C-GAN trains a class-conditional generator that makes target-style labeled samples, with a weight anchor to the source model and a clustering-based smoothness term. Author-reported: 99.4% SVHN to MNIST, 89.6% Office-31 average (ResNet50), 81.6% VisDA17 (ResNet101).Paper · 2025-02-24Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public OpinionsLoRA fine-tuning of LLMs on survey response distributions, so a model predicts how a demographic group answers a multiple-choice question. Trained with forward KL to the group's human distribution on 3,229 Pew questions. Author-reported: Wasserstein distance on OpinionQA falls to 0.094-0.106 from 0.138-0.173 for the best zero-shot baseline.Paper · 2025-02-17A Survey on Bridging EEG Signals and Generative AI: From Image and Text to BeyondSurvey of work that generates images, text and audio/speech from EEG, drawn from 2017-2025 literature. Compares datasets, GAN, VAE, diffusion and transformer decoders, EEG encoders and metrics. Finds small datasets, weak cross-subject generalization and no shared benchmark. Best reported ZuCo EEG-to-text BLEU-1: 52.38 vs 40.10 baseline.Paper · 2025-02-17FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse ViewsA feed-forward transformer for 2 to 8 uncalibrated views. It predicts camera poses first, then camera-centric and global point maps conditioned on those poses, then 3D Gaussians for novel-view rendering, in under 0.5 s. Author-reported: 79.9 AUC@30 on RealEstate10K poses (PoseDiffusion 51.6) and 23.33 dB PSNR on 8-view DL3DV rendering.Paper · 2025-02-09DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot ControlPairs a Qwen2-VL-2B vision-language model with a 1B-parameter diffusion-transformer action expert, trained in three stages (cross-embodiment expert pre-training, embodiment alignment, task post-training). Sub-step text is generated as intermediate output. Author-reported: 0.92 shirt folding without task-specific training, laundry folding 0.4 vs pi0 0.2, LIBERO average 97.3.Paper · 2025-02-04Fully Autonomous AI Agents Should Not be DevelopedPosition paper arguing that fully autonomous AI agents, which write and run their own code beyond predefined constraints, should not be developed. It proposes a five-level autonomy scale and reviews ten ethical values at each level. It has no experiments; the authors argue that risk to people rises with autonomy.Paper · 2025-01-24Humanity's Last ExamHumanity's Last Exam: 2,500 expert-written, closed-ended questions, about 14% with images, built so that frontier LLMs fail and web search does not give the answer. Eight models score 2.7% to 13.4% accuracy with 73% to 89% RMS calibration error (author-reported). Official code: MIT, o3-mini judge.Paper · 2025-01-22Low-dimensional adaptation of diffusion models: Convergence in total variationShows that DDIM and DDPM samplers with their original coefficients reach total-variation error eps in roughly k/eps steps, k an intrinsic dimension of the data, with no smoothness or log-concavity assumed. Extends the bound to learned scores under stated error conditions, gives kernel-estimator rates, and a one-step lower bound says these coefficients are nearly necessary. Theory only.Paper · 2025-01-16FAST: Efficient Action Tokenization for Vision-Language-Action ModelsFAST turns each chunk of robot actions into a short token sequence: a DCT per action dimension, scaling and rounding of the coefficients, then a 1024-entry BPE. This lets autoregressive VLAs learn high-frequency dexterous tasks where per-timestep binning fails. With pi0, it matches diffusion pi0 on the authors' evaluations with about 5x less training compute (author-reported).Paper · 2025-01-15LLM-based Human Simulations Have Not Yet Been ReliablePosition paper arguing that LLM-based human simulations are not yet reliable. Reviews social, economic, policy and psychological uses, attributes the gaps to LLM limits and design flaws, and proposes three solution areas plus a simulate-and-validate loop (Algorithm 1). No experiments of its own; its evidence is cited from prior studies.Paper · 2025-01-06From Tables to Time: Extending TabPFN-v2 to Time Series ForecastingCasts forecasting as tabular regression: time, calendar, seasonal and known-future covariate features per time step, fed to a pretrained TabPFN-v2 (11M parameters) that predicts the whole horizon in one pass, with no time-series pretraining. Author-reported: competitive on GIFT-Eval, state of the art on covariate-informed fev-bench. Cannot extrapolate trends; about 30x slower than two baselines.Paper · 2025-01-01AI-ANNE: (A) (N)eural (N)et for (E)xploration: Transferring Deep Learning Models onto Microcontrollers and Embedded SystemsWorking paper on AI-ANNE, a MicroPython implementation of dense neural networks (neurons, layers, activations) that runs weights exported from TensorFlow/Keras on a Raspberry Pi Pico. Two pre-trained nets (8 and 6 neurons) classify Iris Versicolor vs Virginica at 90% and 95% accuracy on 20 samples. No latency or energy measurements.Paper · 2024-12-30Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social SpacesBuilds SNEI, about 40K human-annotated VQA pairs on 2K robot-human scenes from SCAND, covering perception, prediction, chain-of-thought, action and explanation. Fine-tunes LLaVA-v1.5-7B with LoRA into Social-LLaVA. Human judges (1-5) score it above prompted GPT-4V and Gemini 1.5 Pro on all five task types (author-reported).Paper · 2024-12-30T-DOM: A Taxonomy for Robotic Manipulation of Deformable ObjectsA taxonomy for robot manipulation of deformable objects, with tags for deformation (compression, tension, torsion, shear, structured and unstructured bending), robot motion by energy regime, and grasp and contact. Applied by hand to 10 recorded tasks, it splits actions that the Bullock 2012 and Paulius 2020 taxonomies merge. No learned model; author-reported cluster counts.Paper · 2024-12-29Safe Bayesian Optimization for the Control of High-Dimensional Embodied SystemsHdSafeBO tunes controllers with hundreds to thousands of parameters under a probabilistic safety constraint. It runs a trust-region Thompson-sampling search in a PCA or isometric-VAE latent space, keeping candidates whose optimistic safety bound passes a threshold. Author-reported: objective 3.96 vs 3.07 for the best baseline on 1000-D synthetic functions.Paper · 2024-12-28Embodiment-Agnostic Navigation Policy Trained with Visual DemonstrationsTrains a diffusion policy on hand-held depth-camera demonstrations, with a detected human's relative position as the goal, to output robot-relative waypoints for reaching or following a target while avoiding obstacles. Author-reported on a Unitree Go2: 84% success in the hardest of three levels (25 trials), against 20% for ViNT with a relative-distance goal.Paper · 2024-12-27Hindsight Planner: A Closed-Loop Few-Shot Planner for Embodied Instruction FollowingAn LLM planner for ALFRED household tasks with 140 examples. An actor-critic beam search picks sub-goals from observations and LLM-predicted PDDL arguments. Hindsight prompts relabel the agent's failed rollouts with GPT-4 Turbo so a second actor learns to recover. Author-reported test-seen SR 25.51 and test-unseen SR 18.77, against LLM-Planner at 15.33 and 13.41.Paper · 2024-12-27Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPCReplaces the per-step numerical solve of distributed MPC with explicit closed-loop policies learned online by a distributed actor-critic, with a force-field-style variant for state and control limits. Author-reported: learning and deployment at up to 10,000 simulated robots, a 2-robot policy transferred to 1,000 robots, and wheeled and drone experiments.Paper · 2024-12-25Autonomous Navigation of 4WIS4WID Agricultural Field Mobile Robot using Deep Reinforcement LearningTrains deep RL agents (PPO, SAC, TD3, A2C compared) in Gazebo for a four-wheel-steering, four-wheel-drive farm robot. The agent follows crop rows from camera line detection and moves between waypoints across rows, switching between symmetric, zero-turn and lateral steering. Author-reported: PPO performs best; 351 of 420 simulated trials (83.57%) reach the goal. Simulation only.Paper · 2024-12-23OLiDM: Object-aware LiDAR Diffusion Models for Autonomous DrivingGenerates LiDAR scenes object-first: a point-cloud diffusion model makes foreground objects from a text description and a 3D box, a ControlNet-style controller feeds them into a range-image diffusion model, and an auxiliary per-category loss (OSA) aligns features. Author-reported on KITTI-360: FPD 7.60 vs 25.1 for UltraLiDAR; JSD 0.42 vs 0.71.Paper · 2024-12-23Online Adaptation for Myographic Control of Natural Dexterous Hand and Finger MovementsRegresses 7 wrist and finger degrees of freedom from 16-channel forearm EMG with a causal TCN or LSTM, then keeps updating the model during a free-form session. Author-reported on 9 non-amputees: RMSE 8.0 to 10.4 degrees for TCN and LSTM, against 11.3 to 15.4 for SVR, with near-zero response delay.Paper · 2024-12-21OpenAI o1 System CardSystem card for OpenAI o1 and o1-mini. Reports refusal, jailbreak, hallucination and bias evaluations, a chain-of-thought deception monitor, external red teaming and Preparedness Framework scores. Headline: o1 beats GPT-4o on several jailbreak and refusal sets and is rated medium risk for persuasion and CBRN. The training method is not described; all numbers are author-reported.Paper · 2024-12-20Dexterous Manipulation Based on Prior Dexterous Grasp Pose KnowledgeSplits dexterous manipulation of articulated objects into two stages: an AnyGrasp-based grasp pose for the object's functional part gives the start state, then PPO on point clouds refines it. In XArm6 simulation on four DexArt tasks, success rates beat DexArt (e.g. bucket 94.17% vs 79.17%); real UR5 tests use about 10 trials per task.Paper · 2024-12-19Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement LearningTAR² trains a reward network that splits a sparse episodic team return into per-timestep, per-agent rewards, so single-agent RL (IPPO, MAPPO) can learn from dense signals. The paper claims this is potential-based shaping that keeps the optimal policy; its proof assumes γ=1. On SMACLite 5m_vs_6m it beats other redistribution baselines in a figure; no numbers are given.Paper · 2024-12-19Vibration-based Full State In-Hand Manipulation of Thin ObjectsVibrates a thin object held by a parallel gripper with an eccentric-mass motor on a rotating finger. Moving the centre of mass on a circle rotates the object, then a radial move reaches the goal position. Duty-cycle vibration cuts orientation error (author-reported: 1.5 deg vs 3.8 deg for constant frequency on a disk). Position error is about 1.9 mm.Paper · 2024-12-18A Constraint Embedding Approach for Dynamics Modeling of Parallel Kinematic Manipulators with Hybrid LimbsDynamics model for parallel manipulators whose limbs contain closed loops. Each loop is solved numerically inside its limb, which then acts as a serial chain. A nested Newton-Raphson scheme gives limb inverse kinematics. Author-reported on the 2-DOF IRSBot-2: 1 to 3 iterations per time step, validated against an MBS tool with no error figures.Paper · 2024-12-18Dynamics of Parallel Manipulators with Hybrid Complex Limbs -- Modular Modeling and Parallel ComputingModels parallel manipulators whose limbs contain closed loops, such as Delta robots. Each loop is solved locally, equations of motion use a Lie group screw formulation, and identical limbs reuse one representative model. Author-reported on a Delta example: 2.74 µs for inverse kinematics plus inverse dynamics on one node, and 0.97 µs for the slowest limb when limbs run in parallel.Paper · 2024-12-18Learning Quadrupedal Robot Locomotion for Narrow Pipe InspectionTrains a quadruped robot (Unitree Go2) with PPO in Isaac Gym to cross narrow pipes. The policy gets a bidirectional height scan as privileged input and a reward with centreline and contact penalties. Simulation: 93.75% to 100% success on 0.2 to 0.3 m pipes in stage 3. Real PVC pipes: 100% without obstacles (author-reported).Paper · 2024-12-18Learning from Massive Human Videos for Universal Humanoid Pose ControlUH-1 maps a text command to humanoid motion: a Transformer predicts discrete action tokens, decoded by a VQ-VAE. It is trained on Humanoid-X, 163.8K clips from web and mocap sources retargeted to a robot with RL tracking. Author-reported: FID 0.445 vs 0.582 (MDM) on HumanoidML3D; 70-100% real-robot success per command.Paper · 2024-12-18Planning Human-Robot Co-manipulation with Human Motor Control Objectives and Multi-component Reaching StrategiesPlans robot motion for physical human-robot co-manipulation by optimising a human motor-control objective (speed-accuracy and cost-benefit trade-offs) under Gaussian state uncertainty. The same model estimates the human's goal and predicts when to hand over authority. Simulation shows Fitts-law-like timing; Franka results are qualitative, from one operator.Paper · 2024-12-18SimADFuzz: Simulation-Feedback Fuzz Testing for Autonomous Driving SystemsGenetic fuzzer for autonomous driving in CARLA. A Transformer over simulation feedback scores violation likelihood; NSGA-II selects scenarios by that score, SDC-Scissor, minimum distance and unique violations. Author-reported: 35 unique violations in 6 h on InterFuser, versus 26 (TM-Fuzzer), 18 (DriveFuzz) and 3 (AV-Fuzzer).Paper · 2024-12-18Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall SpacesVSI-Bench: over 5,000 video question-answer pairs from 288 indoor scans, testing eight spatial tasks in multimodal LLMs. Humans score 79.2 on a 400-question subset, the best model 48.8. Text-only reasoning prompts (step-by-step, sampled votes, plan search) do not help. Generating a 10x10 cognitive map raises relative-distance accuracy from 46.0 to 56.0. All numbers author-reported.Paper · 2024-12-17CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement LearningAdds a reward term from a CLIP image-text model to DQN and PPO training of a car crossing an unsignalized intersection in highway-env. CLIP matches each frame against slow-down, proceed and speed-up prompts. Author-reported: CLIP-DQN reaches 96% success with one other vehicle, versus 70% for DQN without CLIP; CLIP-PPO reaches 38%.Paper · 2024-12-16Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial ReasoningEmma-X fine-tunes the 7B OpenVLA policy on BridgeData V2 demonstrations relabelled with Gemini 1.5 Pro. Trajectories are split into segments; the model gets per-segment subtasks and reasoning, and per-frame gripper targets and movement plans. Author-reported on 12 real WidowX tasks: 57.5% success and 71.7% half-success, versus 33.3% success and 45.4% half-success for OpenVLA.Paper · 2024-12-16Visual IRL for Human-Like Robotic ManipulationLearns a reward from RGB-D human keypoints and object positions with adversarial IRL (AIRL), then maps human arm joints to a cobot with a learned neuro-symbolic model and an IK refinement. Author-reported on onion sorting and bottle pouring: learned-behaviour accuracy 90.4% vs 83.3% for discrete state-action input, and shorter, smoother motions than RRT and RRT-connect.Paper · 2024-12-13Distributed Inverse Dynamics Control for Quadruped Robots using Geometric OptimizationComputes quadruped joint torques from full rigid-body inverse dynamics. It enforces the exact friction cone with a custom projected-gradient solver (GPGD), not a generic QP, and splits base and joint tasks by null-space projection. Author-reported on a Unitree Go2: GPGD solves 2.47x faster than qpOASES; hardware base orientation error is 0.0156 rad vs 0.0234 rad for a QP balance controller.Paper · 2024-12-13One Filter to Deploy Them All: Robust Safety for Quadrupedal Navigation in Unknown EnvironmentsAn observation-conditioned reachability value network takes a LiDAR scan, a disturbance bound and the robot state, and predicts an HJ safety value. Its gradient drives a QP filter that overrides a nominal controller when needed. Author-reported: 0.91 success in hard simulation on ABS-Agile (ABS 0.80), and 8/10 successes on a slippery Go1 course where ABS gets 2/10.Paper · 2024-12-13RLDG: Robotic Generalist Policy Distillation via Reinforcement LearningRLDG trains a reinforcement-learning expert per task on a real robot, rolls out its successful episodes and fine-tunes the generalist policies OpenVLA and Octo on them instead of human demonstrations. Author-reported: 10% to 37% higher success on precise real-robot insertion tasks than fine-tuning on human demonstrations, and better generalisation than the RL experts.Paper · 2024-12-13Reward Machine Inference for Robotic ManipulationLearns a reward machine (a finite-state task graph) from unlabelled visual demonstrations of block manipulation. DBSCAN clusters frame features into sub-goal prototypes, which define the propositions and transitions. A DQN on the inferred machine, with potential-based shaping, roughly matches a ground-truth-machine agent on five simulated Panda tasks (author-reported, figures only).Paper · 2024-12-10From Slow Bidirectional to Fast Autoregressive Video Diffusion ModelsTurns a bidirectional video diffusion transformer (about 50 denoising steps) into a 4-step causal generator that produces frames chunk by chunk with KV caching. Distribution matching distillation with a bidirectional teacher supervises the causal student, after ODE-pair initialization. Author-reported VBench-Long total 84.27 and 9.4 FPS streaming on one GPU.Paper · 2024-12-10Optimizing Locomotor Task Sets in Biological Joint Moment Estimation for Hip Exoskeleton ApplicationsClusters 20 locomotor tasks by PCA and k-means on hip kinematics and moments, then picks 8 representative tasks by a task-weight score. A small FCNN trained on those 8 tasks estimates hip joint moment with leave-one-subject-out RMSE 0.30 ± 0.05 Nm/kg, close to all tasks (0.28) and better than cyclic tasks only (0.38). Author-reported, 11 healthy subjects.Paper · 2024-12-10SALON: Self-supervised Adaptive Learning for Off-road NavigationSALON learns off-road traversability online from the robot's own driving. It maps foundation-model features into a bird's-eye-view grid, pairs them with bandpower roughness, and fits a Gaussian process. One user click marks a lethal object. Author-reported: comparable navigation to methods trained on 100-1000x more data.Paper · 2024-12-09Haptics in Micro- and Nano-ManipulationBilateral haptic tele-manipulation of magnetically guided microrobots: operator motion is scaled down to the tool, and interaction forces are scaled up to a haptic device. Gives a dynamics model and a PI force-feedback controller, tested in three single-trial experiments. Headline: 8 µm steady-state position error for a microbead. The stability analysis promised in the abstract is absent.Paper · 2024-12-07SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of ExpertsPuts a mixture-of-experts layer in the cross-attention query of the DUET navigation agent and routes each step by the pooled visual views plus the instruction [CLS] token, so one model handles seven vision-language navigation tasks. Trained with DAgger and a load-balance loss (weight 0.8). Author-reported: SR 76 on R2R val unseen, 46.4 on REVERIE val unseen, 76.3 on ObjectNav MP3D val.Paper · 2024-12-06Get It Right: Improving Comprehensibility with Adaptable Speech Expression of a Humanoid Service RobotDescribes an Android/Kotlin app for a Pepper humanoid in a public-service office. It answers questions from a knowledge base and, on request, restates the answer in German easy language (SUMM AI) or in Danish (DeepL). Case study only: no measurements; workshop feedback with public-service staff is described as positive.Paper · 2024-12-06Talking Like One of Us: Effects of Using Regional Language in a Humanoid Social RobotUser study (within-subject, 17 participants) of a Pepper robot giving the same dialogue in High German and in Low German, rated on the RoSAS scale. Low German raised perceived warmth (p = 0.027); competence (p = 0.378) and discomfort (p = 0.056, slightly lower) did not reach significance.Paper · 2024-12-05Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image SynthesisInfinity generates images scale by scale, predicting bit labels from a binary tokenizer with a 2^32 vocabulary. A per-bit two-way head replaces a huge softmax, and bitwise self-correction flips training bits so the model learns to fix its own errors. Author-reported: GenEval 0.73 with prompt rewriting and DPG 83.46 for a 2B model, with 0.8 s per 1024x1024 image.Paper · 2024-12-05LiveNet: Robust, Minimally Invasive Multi-Robot Control for Safe and Live Navigation in Constrained EnvironmentsNeural controller for two robots meeting in doorways or intersections, trained by imitating a receding-horizon MPC. It outputs a reference command and learned penalties for a collision and a liveness control barrier function; a QP layer changes only speed. Author-reported in simulation: no collisions or deadlocks in 50 runs per scenario; baselines take 10-20 times longer per step.Paper · 2024-12-05TANGO: Training-free Embodied AI Agents for Open-world TasksTANGO composes a pretrained PointGoal navigation policy, a VLFM-style value map with a feature memory, and vision detectors into primitives. An LLM writes a short program from 15 in-context examples, with no training. Author-reported zero-shot results: OVON val-unseen SR 35.5 and SPL 19.5, OpenEQA score 37.2, GOAT-Bench SR 32.1 and SPL 16.5.Paper · 2024-12-04A Teleoperation System with Impedance Control and Disturbance Observer for Robot-Assisted RehabilitationBilateral teleoperation for two 2-DOF planar rehabilitation robots. Impedance control with a nonlinear disturbance observer (NDOB) runs on both sides, and the master renders a virtual-spring force from the position error alone. Supports pre-defined trajectories, free pHRI and record-replay. Evidence is qualitative: trajectory plots, no tracking-error numbers, one author.Paper · 2024-12-04Design and Control of an Ultra-Slender Push-Pull Multisection Continuum Manipulator for In-Situ Inspection of AeroengineA tendon-driven continuum manipulator with three push-pull sections and nine tendons, for inspecting aeroengine blades through one port. Each section uses a constant-curvature kinematic map, a coordinate rotation decouples the tendon groups, and a PID tension term keeps tendons taut. Author-reported: length-to-diameter ratio up to 62.75; qualitative decoupling and inspection results only.Paper · 2024-12-04Real-time Dynamics of Soft Manipulators with Cross-section Inflation: Application to the Octopus Muscular HydrostatExtends Cosserat rod dynamics with a cross-section inflation ratio, a Saint-Venant Kirchhoff elastic law with Kelvin-Voigt damping, and a reduced-order Geometric Variable Strain solver. Tested on stiffness tuning and octopus-like reaching. Author-reported: arm elongation of about 20% of the bend-point travel and volume change within 3%, where the classic Cosserat model loses volume.Paper · 2024-12-04Resilient Timed Elastic Band Planner for Collision-Free Navigation in Unknown EnvironmentsRTEB extends the Timed Elastic Band local planner. When the TEB trajectory is infeasible, a hybrid A* search with a novel heuristic finds a car-like path, a Voronoi-field soft-constraint smoother refines it, and TEB is reinitialised. Author-reported: about 20% lower distance, time and control effort than TEB and NMPC in dense obstacle settings (ROS2, real and Gazebo).Paper · 2024-12-03Generating Critical Scenarios for Testing Automated Driving SystemsAVASTRA trains a Double DQN agent with prioritized replay to set LGSVL scene parameters (weather, time of day, NPC vehicles, pedestrians) so that Apollo 7.0 collides. Rewards use a safety-distance collision probability. Author-reported: 11.6 to 18.9 collisions per road, versus 5.4 to 12.6 for DeepCollision.Paper · 2024-12-03Haptic Stiffness Perception Using Hand Exoskeletons in Tactile Robotic TelemanipulationTen participants squeezed five soft silicone objects through an Allegro hand with tactile fingertips and an HGlove exoskeleton, without visual feedback. Method I maps contact force to glove force; Method II also scales it by a leader-follower displacement ratio. ABX success averaged about 75% for both methods; Method II may help for similar-stiffness pairs, with no consistent overall gain.Paper · 2024-12-03Improving Dynamic Object Interactions in Text-to-Video Generation with AI FeedbackOffline RL-finetuning of a 3B text-to-video diffusion model on prompts of dynamic object interactions, rewarded by binary accept/reject judgments from a vision-language model. Reward-weighted regression and DPO both beat the pre-trained and SFT models, and AI feedback gives the best Gemini, GPT-4o and human scores. Results are author-reported on 40 test prompts.Paper · 2024-12-03LLM-Enhanced Path Planning: Safe and Efficient Autonomous Navigation with Instructional InputsAn LLM reads a natural-language instruction and turns it into grid-map actions (reset the map, change costs in a region, set a goal) that the planner then executes, replanning as the map changes. In one Gazebo Husky-UR3 pick-and-place simulation, a Llama3.1 variant expanded 370 nodes against 2,254 for A* in quick mode. The paper reports 90% instruction-to-task accuracy. Author-reported.Paper · 2024-12-03Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation ApplicationsPrithvi-EO-2.0 is a ViT masked autoencoder (300M and 600M parameters) pretrained on 4.2M four-frame 30 m HLS satellite samples. It uses 3D patch embeddings and optional date and location metadata. Author-reported: 8% over Prithvi-EO-1.0 on GEO-Bench and strong results on SME-led downstream tasks.Paper · 2024-12-03SparseGrasp: Robotic Grasping via 3D Semantic Gaussian Splatting from Sparse Multi-View RGB ImagesFits a 3D Gaussian Splatting scene to a few RGB views, initialised from DUSt3R points, and distils PCA-compressed MaskCLIP region features into it. A render-and-compare step updates moved objects in about 200 ms; a retrained GraspNet grasps from the Gaussians. Real arm, 3 views: 55 of 70 grasps, vs 38 of 70 for F3RM with 17 views (author-reported).Paper · 2024-12-01STEVE-Audio: Expanding the Goal Conditioning Modalities of Embodied Agents in MinecraftAdds audio prompts to the Minecraft agent STEVE-1 without retraining its policy. A new Audio-Video CLIP model (frozen MineCLIP video and AST audio encoders plus trainable mapping networks) and a CVAE prior map audio into STEVE-1's goal space. Author-reported: audio beats the visual prompt on 4 of 6 item-collection tasks and the text prompt on 5 of 6.Paper · 2024-11-30Dynamics Modeling using Visual Terrain Features for High-Speed Autonomous Off-Road DrivingHybrid off-road vehicle model: a bicycle-type physics model plus an LSTM that corrects its forces from DINOv2 terrain features, compressed by an encoder trained end-to-end. Author-reported: about 10% lower mean summed loss than a no-vision model with hindsight features. Features from realistic projection distances help only within 0 to 30 m.Paper · 2024-11-30Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query PerspectiveSurvey of knowledge graph reasoning organised by query type: single-hop completion with symbolic, neural and neural-symbolic methods; complex first-order logical queries; natural-language single- and multi-turn questions; and combinations with LLMs. It proposes no method and reports no experiments, so it is a map of method families rather than a source of numbers.Paper · 2024-11-29Grasping and Rolling In-plane Manipulation Using Deployable Tape spring AppendagesGRIP-tape is a two-arm gripper whose arms are bidirectional tape springs: two tape-measure blades bonded together. The arms roll into a compact spool and extend into beams. Seven motors set each arm's length and angle and the gap. Shows grasping, in-hand rolling, conveyance, base load-cell force sensing and automatic grip. Author-reported: 3.72 ± 0.35 mm mean tip error over 16 targets.Paper · 2024-11-28SoGraB: A Visual Method for Soft Grasping Benchmarking and EvaluationSoGraB scores a soft-gripper grasp without force sensors. It compares 3D point clouds of the object before and after grasping, aligns them with ICP, and measures deformation with Density-Aware Chamfer Distance (DCD). That is combined with success and drop time into a 0 to 1 score. Author-reported: 900 grasps on 15 objects, 3 hardnesses and 4 grippers, scores from 0.517 to 0.940.Paper · 2024-11-26DexGrip: Multi-modal Soft Gripper with Dexterous Grasping and In-hand Manipulation CapacityDexGrip is a soft gripper with three Fin Ray fingers, each with a motor-driven belt, and a telescoping palm with a rotating suction cup (9 DoFs). In palm tests on 12 objects, 9 succeed overall. Pinch-to-power and cube re-orientation are shown qualitatively. No official code found.Paper · 2024-11-25Picking by Tilting: In-Hand Manipulation for Object Picking using Effector with Curved FormPicks up an object too large to grasp by tilting it against two flat supports with a curved, passive palm, then tucking the palm under it. Gives two tilt modes and a force-closure planning check in (theta, delta). Author-reported: a 2-DOF palm succeeds 4/5 to 5/5 trials per object; a UR3 arm fails on a steel box at 90 degrees (0/5).Paper · 2024-11-25RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for RoboticsGenerates about 3M spatial questions from 3D indoor and tabletop scans: free-space context points, whether an object fits, and left/right/above/front relations in ego, world and object frames. Fine-tuning 2D and 3D VLMs on it lifts held-out accuracy (LLaVA-NeXT 30.3 to 60.5). Author-reported.Paper · 2024-11-23A Survey on LLM-as-a-JudgeSurvey and taxonomy of LLM-as-a-Judge: formal definition, prompt and model choices, post-processing, reliability strategies (fine-tuning, feedback, multi-round and multi-LLM voting) and meta-evaluation. A small six-LLM experiment finds GPT-4-turbo least biased and 5-round majority voting with position swapping most useful.Paper · 2024-11-23Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot InteractionCompares one LLM agent (GPT-4) with two multi-agent setups that write Python for a Boston Dynamics Spot robot from 7 spoken prompts. The coder-plus-reviewer setup made the fewest first-attempt code errors (9 points below the single agent). Observer scores showed no significant difference across configurations. Author-reported, 3 runs.Paper · 2024-11-22Aim My Robot: Precision Local Navigation to Any ObjectAim My Robot (AMR) drives a robot to a centimetre-precise pose relative to any object, given a reference image with an object mask and a relative side, distance and angle. A transformer fuses frozen MAE RGB-D tokens, LiDAR bins and goal tokens, and predicts waypoints autoregressively. Trained on 500k simulated trajectories. Reports a 3 cm and 1 degree median error on unseen objects.Paper · 2024-11-22FastGrasp: Efficient Grasp Synthesis with DiffusionFastGrasp generates a human hand pose for an object point cloud in one network pass. A latent diffusion model over a hand-pose autoencoder is followed by an MLP adaptation module trained with contact and penetration losses. Author-reported: lowest penetration volume among compared methods on out-of-domain HO-3D (5.23) and 0.14 s inference on an A40.Paper · 2024-11-21Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMsA small imitation-learning driving model (ViT-B/16 with per-class steering and throttle heads) is paired with a frozen multimodal LLM that is prompted with chain-of-thought and returns LEFT, MIDDLE or RIGHT at its own slower rate. Trained on one cone, the RC car on a phone reaches 40% success alone and 63-83% with an LLM on a harder real hallway (author-reported, 30 runs).Paper · 2024-11-21Open Challenges in the Formal Verification of Autonomous DrivingPosition paper. A driver-monitoring and smart-city sensing case study names two open challenges: composing black-box probabilistic components, and mapping software onto many-core hardware with redundancy and voting. Suggests assume-guarantee plus runtime verification as directions. No implementation or experiments; no quantitative results.Paper · 2024-11-21Understanding World or Predicting Future? A Comprehensive Survey of World ModelsSurvey of world models: learned models that build internal representations of how the world works or predict its future states. Covers model-based RL, LLM world knowledge, video generators, embodied environments, and applications in games, driving, robotics and social simulation, plus open problems. No experiments of its own; the key formulas are the model-learning and MPC planning objectives.Paper · 2024-11-20Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidanceHierarchical Diffusion Policy (HDP) splits imitation learning into two diffusion models. A Guider predicts the next object contact; an Actor generates a 16-step action sequence toward it, trained with behaviour cloning plus a Q-learning term. Author-reported: 20.8% average success gain over Diffusion Policy across six tasks.Paper · 2024-11-19Locomotion Mode Transitions: Tackling System- and User-Specific Variability in Lower-Limb ExoskeletonsPersonalizes a threshold-based finite-state detector of walk-to-sit and stair transitions on two lower-limb exoskeletons (eWalk, autonomyo). Two methods adapt the thresholds: a statistics-based rescaling and Bayesian optimization on labelled user data. A linear hip-angle mapping is fitted for autonomyo. Author-reported: up to 80% gain over non-personalized thresholds in some scenarios.Paper · 2024-11-18InstruGen: Automatic Instruction Generation for Vision-and-Language Navigation Via Large Multimodal ModelsSamples trajectories from YouTube house-tour videos, labels rooms with CLIP and actions with an action model, then has ChatGPT-4V write R2R-style instructions. A consistency check and regex rules filter the text. A ViLBERT-like agent is pretrained on the pairs. Author-reported: R2R val-unseen SR 71.10 (Lily 70.00), RxR zero-shot SR 29.35. The linked repository holds only a README.Paper · 2024-11-16Bias in Large Language Models: Origin, Evaluation, and MitigationA review of bias in LLMs. It splits bias into intrinsic (in pretrained representations) and extrinsic (in downstream tasks and outputs), groups evaluation methods into data-, model- and output-level, and groups mitigation into pre-model, intra-model and post-model. It runs no experiments of its own; its cost comparison is an estimate.Paper · 2024-11-15'What did the Robot do in my Absence?' Video Foundation Models to Enhance Intermittent SupervisionMakes generic or query-driven summaries of 40-minute robot videos as storyboards, short videos or text, using pretrained video models (STVT, LanguageBind, VideoChat2) without fine-tuning. In a 30-person study, query-driven summaries raised answer accuracy over raw video but added task time; storyboards were the most accurate modality, weakest on event questions. Author-reported.Paper · 2024-11-15Moving Forward: A Review of Autonomous Driving Software and Hardware SystemsSurvey of autonomous-driving sensors, datasets, modular and end-to-end software, and CPU/GPU/SoC hardware. It profiles TransFuser, InterFuser and MILE on four CPU/GPU systems. Author-reported: no CPU alone reaches the 10 FPS minimum except one borderline case, GPUs give more FPS per watt, and the authors argue for task-specific and processing-in-memory hardware.Paper · 2024-11-14Modular Fault Diagnosis Framework for Complex Autonomous Driving SystemsA modular fault diagnosis framework for autonomous driving. Modules report OK, WARNING, ERROR, IGNORE or UNKNOWN per subsystem. Aggregators group them, and dependency and driving-state rules mark affected parts IGNORE to show the root cause. Run on FZI shuttle buses in six qualitative fault scenarios; no metrics, no code.Paper · 2024-11-12Effective Virtual Reality Teleoperation of an Upper-body Humanoid with Modified Task Jacobians and Relaxed Barrier Functions for Self-Collision AvoidanceFour off-the-shelf 6-DoF VR trackers (headset, two hands, waist) drive a 17-DoF humanoid. Each tracker is tied to a fixed joint set by removing unassigned joints from its task Jacobian. Self-collision is a soft relaxed-barrier term in a QP-based IK solve. Results are qualitative only: a table-top box-packing and two-handed handover demo.Paper · 2024-11-12Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-LearningQPHIL quantizes states into learned landmarks with a VQ-VAE, trained with a temporal contrastive loss. A transformer plans a landmark sequence, and two IQL policies (landmark and goal) follow it. Author-reported: best success on AntMaze-Ultra and up to 50% on a new AntMaze-Extreme; competitive on smaller mazes.Paper · 2024-11-12Point Cloud Context Analysis for Rehabilitation Grasping AssistanceA wrist-mounted depth camera and IMU classify a scene as tabletop, doorknob or pot handle from RANSAC plane detection and clustering on point clouds, then propose opposing grasp points via PCA. Author-reported: 85-100% mode accuracy on bench scenes and grasp points within about 5-6 mm for one eraser-sized object.Paper · 2024-11-09Grasping Object: Challenges and Innovations in Robotics and Virtual RealityA survey of grasping in virtual reality, compared with robotic grasping. It lists the open challenges (deformable bodies, hand modelling, device limits, no common metric) and groups methods into physics-based, visual and data-driven. It proposes no new method and reports no experiments.Paper · 2024-11-09Research on reinforcement learning based warehouse robot navigation algorithm in complex warehouse layoutProximal policy-Dijkstra (PP-D): Dijkstra's algorithm plans a shortest path on the static warehouse layout, then PPO learns an action policy that adapts to changes in the environment. Author-reported on a single test set against DQN and A*: accuracy 96.42%, recall 95.83%, F1 94.88%, robustness 75.62%. Setup, training details and baseline implementations are not stated.Paper · 2024-11-08End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-AnsweringVLMnav turns an off-the-shelf VLM into a zero-shot navigation policy. A depth-based top-down map proposes collision-free polar actions, which are drawn as numbered arrows on the RGB image. The VLM picks one arrow per step, and a second VLM call decides when to stop. Author-reported: 50.4% SR and 0.210 SPL on Habitat ObjectNav with Gemini Flash, against 24.6% and 0.106 for PIVOT.Paper · 2024-11-07Field Assessment of Force Torque Sensors for Planetary Rover NavigationField test of six force-torque sensors, one above each wheel of the MaRTA planetary rover, on four terrain types. A neural network on sensor statistics reaches 96.17% test accuracy (IMU only: 79.35%). Estimating drawbar pull from the sensors by lever-length filtering is not reliable enough without ground truth. Author-reported; MIT code.Paper · 2024-11-07Raising Body Ownership in End-to-End Visuomotor Policy Learning via Robot-Centric PoolingRobot-centric Pooling (RcP) replaces the final pooling of a ResNet18 visuomotor policy. A cross-attention query built from the joint and end-effector state picks the image region that matches the robot, and a MoCo-style contrastive loss trains it jointly with an L1 policy head. Author-reported: 76.4% success under unseen distractors in simulation, against 31.9% (Spatial-Softmax) and 8.3% (R3M).Paper · 2024-11-06From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement LearningStepAgent gives LLM agents a step-wise reward without human step labels. It pairs each expert action with the agent's own action at the same prefix, then trains with a DPO-style implicit reward or an inverse-RL discriminator. On Mistral-7B and Llama-3-8B it beats the open-model baselines on all seven benchmarks (author-reported).Paper · 2024-11-06Object-Centric Dexterous Manipulation from Human Motion DataA transformer planner, trained by behavior cloning on ARCTIC human wrist motion, proposes wrist poses for a goal object trajectory. PPO trains a finger controller that adds a bounded wrist residual; augmentation rounds feed successes back. Author-reported: 83.8% completion on trained single-object trajectories in simulation (59.6% without augmentation); 64.3-100% on six real objects.Paper · 2024-11-05Digital Twin for Autonomous Surface Vessels: Enabler for Safe Maritime NavigationA review that maps digital twins of autonomous surface vessels onto capability levels 0 to 5, from standalone to autonomous. For each level it surveys enabling technologies: CAD and simulators, sensors and fusion, condition monitoring, forecasting and control. It runs no experiments; its result is a level-by-level map of existing enablers.Paper · 2024-11-04Enhancing Social Robot Navigation with Integrated Motion Prediction and Trajectory Planning in Dynamic Human EnvironmentsAdds Social GAN human-trajectory forecasts to the Timed Elastic Band (TEB) local planner. The TEB cost gets terms built from predicted human and obstacle positions and a priority-side term. Author-reported, in Gazebo simulation: MP-TEB has the shortest total time in 3 of 4 scenarios and the largest minimum human-robot distance in 3 of 4 (Table 1).Paper · 2024-11-03Learning World Models for Unconstrained Goal NavigationMUN trains a goal-conditioned Dreamer-style world model on episodes that travel between key subgoal states picked from the replay buffer (DAD: farthest-point sampling over actions), so the model learns backward and cross-trajectory transitions. Author-reported: >95% success on 3-block stacking vs about 60% for baselines, and better navigation between arbitrary subgoals on three tasks.Paper · 2024-11-02The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid ControlTrains diffusion policies on walking data from an AMP-trained RL policy for a simulated Unitree H1 in IsaacGym. It varies the randomization and dataset size (500K to 8M transitions). Terrain and perturbation randomization give the most robust policies, and walking needs far more data than manipulation. Author-reported: the 8M "All" policy matches the source RL policy on a non-randomized target.Paper · 2024-11-01Mixed Reality Teleoperation Assistance for Direct Control of HumanoidsTeleoperation assistance for the Nadia humanoid. Learned ProMPs from simulated demonstrations are conditioned on the operator's first motion and object pose; an Affordance Template takes over the grasp via a logistic blend. Author-reported: 2.35 cm RMS left-hand error on punching; in an eight-person door-opening study, assisted control completed 100% of trials vs 58-75% direct.Paper · 2024-10-31YOLO Evolution: A Comprehensive Benchmark and Architectural Review of YOLOv12, YOLO11, and Their Previous VersionsBenchmarks 33 Ultralytics YOLO checkpoints (YOLOv3u to YOLOv12) fine-tuned with one shared setting on traffic signs, African wildlife and ships. Reports precision, recall, mAP50, mAP50-95, per-image time, GFLOPs and size. Author-reported: YOLO11 balances accuracy and speed best; YOLOv12 is slower with no clear accuracy gain. The linked repo holds traffic-sign notebooks only.Paper · 2024-10-29Robot Policy Learning with Temporal Optimal Transport RewardTemporalOT adds frame order to the optimal-transport (OT) proxy reward for learning robot policies from one or two expert demos. A context window in the cost and a banded temporal mask in a masked Sinkhorn plan make the reward order-aware. On 9 Meta-World tasks with 2 demos, average success is 61.1 versus 35.9 for ADS (author-reported).Paper · 2024-10-25GPT-4o System CardSystem card for GPT-4o, an omni model that takes text, audio, image and video and outputs text, audio and image. It reports speech-focused safety work (voice cloning, speaker identification, bias tests), Preparedness scores (overall medium, persuasion borderline), third-party METR and Apollo results, and text capability gains. Author-reported; no code or weights released.Paper · 2024-10-25In-Simulation Testing of Deep Learning Vision Models in Autonomous Robotic ManipulatorsMARTENS searches an Isaac Sim pick-and-place scene with a genetic algorithm over box poses and lighting for failures of a YOLOv8 oriented-box detector, then fine-tunes the detector on them. Author-reported: 50% (UC-1) and 26% (UC-2) more failures than random search, over 99% repaired, real-world mAP 0.948 and 0.892 after fine-tuning.Paper · 2024-10-24Breaking Down the Barriers: Investigating Non-Expert User Experiences in Robotic Teleoperation in UK and JapanUser study of 74 non-experts in the UK and Japan teleoperating three robot arms through the TELESIM digital-twin framework, stacking three cubes for 10 minutes. The UR5e built the most towers with the lowest mental workload, and the Japanese group reported more trust in robots (NARS). Author-reported; robot, country and recruitment are confounded.Paper · 2024-10-24Comparing YOLOv11 and YOLOv8 for instance segmentation of occluded and non-occluded immature green fruits in complex orchard environmentBenchmarks YOLO11-seg and YOLOv8-seg in five sizes (n to x) on 991 annotated orchard images of immature green apples, split into occluded and non-occluded fruit. Under one training recipe, YOLO11m-seg has the best mAP@50 overall (box 0.876, mask 0.860); YOLOv8n-seg has the lowest inference time (3.3 ms). Author-reported, one run per model.Paper · 2024-10-23YOLOv11: An Overview of the Key Architectural EnhancementsArchitectural overview of YOLOv11: C3k2 blocks replace C2f, a C2PSA spatial-attention block follows SPPF, and one model family spans nano to xlarge for detection, segmentation, pose, OBB and classification. A review with no experiments of its own; its headline gains (22% fewer parameters than YOLOv8m, about 54.5% COCO mAP50-95 for YOLOv11x) come from third-party sources.Paper · 2024-10-16Leveraging LLMs for Translating and Classifying Mental Health DataClassifies depression severity (4 levels) in Reddit posts with GPT-3.5-turbo at temperature 0, first in English, then after the same model translates the posts into Greek. Author-reported macro F1 is 0.17 in English and 0.16 in Greek, with the English run as the only baseline. The authors conclude that LLM translation and classification need careful checks before use in healthcare.Paper · 2024-10-16ProSA: Assessing and Understanding the Prompt Sensitivity of LLMsDefines PromptSensiScore (PSS), the average score gap across the prompt variants of one instance, and measures it for eight LLMs on four objective and two subjective benchmarks. Larger models and few-shot examples lower PSS; higher decoding confidence goes with lower PSS. Author-reported.Paper · 2024-10-14Incorporating Task Progress Knowledge for Subgoal Generation in Robotic Manipulation through Image EditsTaKSIE generates the next visual subgoal image for a language-conditioned robot task. A diffusion model conditioned on the current view, the instruction and a GRU over past keyframes is trained on subgoals chosen by a progress signal; a progress check decides when to generate the next one. Author-reported: CALVIN average length 3.18 (SuSIE 2.80); real-robot success 71.9% (SuSIE 56.3).Paper · 2024-10-09MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific HypothesesSplits chemistry hypothesis generation into three GPT-4o steps: screen a literature corpus for inspirations, compose hypotheses from the background and each inspiration through an evolutionary mutate-and-recombine loop, and rank them by self-scores. On a 51-paper benchmark, the top hypothesis reaches a Matched Score of 4.02 against 2.88 for the MOOSE baseline (author-reported, GPT-4o judge).Paper · 2024-10-03Tutor CoPilot: A Human-AI Approach for Scaling Real-Time ExpertiseTutor CoPilot shows tutors, during live math chat sessions, several expert-style reply suggestions, one per pedagogical strategy, generated by GPT-4 with the Bridge method. In a randomized trial of 783 tutors, students of tutors with access passed exit tickets 4 p.p. more often (62% to 66%, p<0.01). Author-reported.Paper · 2024-10-02Provably Accurate Shapley Value Estimation via Leverage Score SamplingLeverage SHAP estimates Shapley values by weighted least squares over sampled feature subsets, drawn with leverage scores that have a closed form, so no subset list is built. Author-reported: about 50% lower error than the SHAP library's Optimized Kernel SHAP, averaged over 5n to 160n evaluations on eight datasets.Paper · 2024-09-29A Review on Edge Large Language Models: Design, Execution, and ApplicationsSurvey of running LLMs on edge devices. It groups the field into offline compression (quantization, pruning, distillation, low-rank, compact designs), runtime optimizations (cloud-edge collaboration, single-device scheduling, frameworks, hardware co-design, CPU/GPU/NPU) and on-device applications. It runs no experiments of its own.Paper · 2024-09-29OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing ImagesTransformer detector for oriented boxes in remote-sensing images. Boxes become Gaussians for the positional encoding; decoder self-attention gets Gaussian Wasserstein distance biases; cross-attention samples features at points rotated by box angle. Author-reported ResNet-50 AP50: 67.28 on DIOR-R, 75.37 on DOTA-v1.0 (+1.16 and +1.21 over ARS-DETR).Paper · 2024-09-27Evaluation of OpenAI o1: Opportunities and Challenges of AGIEvaluates OpenAI's o1-preview on 27 tasks across reasoning, planning, creation, diagnosis and reflection, using public or private datasets and author-written prompts. Headline results: 10 of 12 LeetCode problems passed (83.3%), best ROUGE-1 and ROUGE-L on 10 radiology reports, 7 of 12 college math answers correct. Samples are small, grading is by the authors, and no code is released.Paper · 2024-09-22Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic ManipulationScales a transformer Diffusion Policy (ScaleDP) from 10M to 1B parameters for visuomotor robot control. It replaces cross-attention observation conditioning with adaptive layer norm (AdaLN) and removes the causal mask in self-attention. Author-reported: the 1B model averages 87.5% on four Franka tasks (DP-T 51.25%) and 98.33% on three bimanual UR5 tasks (DP-T 23.33%).Paper · 2024-09-16PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit RemeshingReconstructs a textured 3D human mesh from one full-body photo. A cross-scale multiview diffusion model, conditioned on SMPL-X renders, generates six body views and a face view as normal and color maps; remeshing then carves an SMPL-X-initialised mesh to them. Author-reported: lower Chamfer distance than baselines on CAPE and THuman2.1.Paper · 2024-09-10What is the Role of Small Models in the LLM Era: A SurveyA survey, not a method. Maps how small models and LLMs collaborate (distillation, data curation, RAG, cascading, routing, speculative decoding, LLM evaluation) and where small models compete or complement them (edge, low latency, narrow tasks, interpretability). Its one own experiment is an MTEB comparison shown only in figures.Paper · 2024-09-09DexDiff: Towards Extrinsic Dexterity Manipulation of Ungraspable Objects in Unrestricted EnvironmentsGrasps large flat objects by first pushing them against a wall or table edge. A fine-tuned vision-language model (VisualGLM-6B) writes the plan; a transformer diffusion policy (GCAD), conditioned on the plan's return-to-go values, predicts short action chunks. Author-reported: 70% average simulated success vs 23% for Diffusion Policy-T across four scenes, and 65% on a UR5e.Paper · 2024-09-05DRAL: Deep Reinforcement Adaptive Learning for Multi-UAVs Navigation in Unknown Indoor EnvironmentTrains a drone policy with PPO and an asymmetric actor-critic (privileged state in training, partial observations at deployment) so that three cable-slung hex-rotors can find and carry an unknown payload through a simulated indoor room with one camera. Author-reported Gazebo results: success 0.824 to 0.903 versus 0.502 to 0.786 for Q-learning, DQN and Sarsa.Paper · 2024-09-03A Modern Take on Visual Relationship Reasoning for Grasp PlanningD3G detects objects with a DETR-style transformer and predicts an adjacency matrix of grasp dependencies between object queries with a dense graph transformer, trained with Hungarian matching and a BCE loss on matched pairs. On the new D3GD bin-picking benchmark it reports the best AP_rel in every synthetic split and 24.44 AP_rel on synth-to-real (author-reported).Paper · 2024-09-01Diffusion Policy Policy OptimizationDPPO fine-tunes a pre-trained diffusion policy with PPO. It treats each denoising step as a Gaussian policy inside a larger MDP, so the policy gradient reaches every denoising step. Author-reported: over 90% success on Robomimic Transport from state, and 80% (16/20) zero-shot sim-to-real on FurnitureBench One-Leg, where a fine-tuned Gaussian policy gets 0%.Paper · 2024-08-28What is YOLOv8: An In-Depth Exploration of the Internal Features of the Next-Generation Object DetectorDescriptive review of YOLOv8 (Ultralytics, 2023): CSP-style backbone, FPN+PAN neck, anchor-free head, mosaic and mixup augmentation, focal, IoU and objectness losses. Runs no experiments of its own. Reprints a YOLOv5 vs YOLOv8 table (mAP@0.5 50.5% vs 55.2%) and five model sizes (n to x) without dataset, hardware or setup. Few training hyperparameters are stated.Paper · 2024-08-28microYOLO: Towards Single-Shot Object Detection on MicrocontrollersmicroYOLO is a YOLO-style single-shot detector for Cortex-M7 microcontrollers. It takes 128x128 RGB input, uses a small depthwise-separable backbone, then L1 pruning and 8-bit quantization. On an OpenMV H7 R2 it runs at about 3.5 FPS in under 800 KB Flash and 350 KB RAM. Author-reported mAP50 is 56.4% (fridge task).Paper · 2024-08-26Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous DrivingDrive-OccWorld is a camera-only world model that forecasts future 4D occupancy and flow from BEV features. It normalises its memory by ego motion, takes actions such as velocity and command, and plans by picking the candidate trajectory with the lowest occupancy cost. Author-reported: 36.3 mIoU for future movable-object occupancy at 2 s on nuScenes, versus 26.8 for Cam4DOcc.Paper · 2024-08-20OMEGA: Efficient Occlusion-Aware Navigation for Air-Ground Robot in Dynamic Environments via State Space ModelOMEGA pairs OccMamba, a LiDAR-to-3D-occupancy network with Mamba blocks in separate semantic and geometric branches fused in BEV, with an ESDF-free planner for air-ground robots. Author-reported: OccMamba 25.0 mIoU and 59.9 IoU on the SemanticKITTI test set; 96% planning success in five real dynamic scenes.Paper · 2024-08-20RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot HandsRP1M is a simulated dataset of about one million piano-playing trajectories from two Shadow Hands, one RL specialist per piece. Finger placement is an optimal-transport assignment, so no human fingering labels are needed. Author-reported: OT fingering matches human-fingering RL on two songs; DP-U reaches F1 0.539 in-distribution (12 songs), DP-T 0.316 on unseen songs (150).Paper · 2024-08-14The AI risk repository: A meta-review, database, and taxonomy of risks from artificial intelligenceSystematic meta-review of 74 AI risk frameworks, extracting 1,725 risks into a living database organised by two taxonomies: a Causal Taxonomy (entity, intent, timing) and a Domain Taxonomy (7 domains, 24 subdomains). Headline: AI systems are the cause of 42% of coded risks and humans of 38%; 62% of coded risks are post-deployment.Paper · 2024-08-12The AI Scientist: Towards Fully Automated Open-Ended Scientific DiscoveryAn LLM loop that proposes research ideas, checks novelty with a literature search, runs experiments through the Aider coding agent, writes a LaTeX paper and reviews it. Author-reported: about $10-15 per paper; the best calibrated reviewer reaches 0.65 balanced accuracy versus 0.66 for NeurIPS human reviewers.Paper · 2024-08-12Towards Robust Monocular Depth Estimation in Non-Lambertian SurfacesFine-tunes Depth Anything V2 Large on Hypersim with three additions: a gradient-domain loss on transparent and mirror (ToM) regions, random tone-mapping augmentation, and optional VAE fusion of exposures at test time. Author-reported: ToM delta1.05 of 0.859 vs 0.644 for Depth Anything V2 on Booster (+33.39%), and 0.606 vs 0.576 on NYU Mirror3D (+5.21%).Paper · 2024-07-31Gemma 2: Improving Open Language Models at a Practical SizeGemma 2 is a family of 2B, 9B and 27B decoder-only models that alternate local sliding-window and global attention, soft-cap logits and use GQA. The 2B and 9B models are trained by distilling a larger teacher. Author-reported: 27B is best in its size class on the HF suite and 9B averages 70.2 on eight benchmarks.Paper · 2024-07-30Learning Multi-Modal Whole-Body Control for Real-World Humanoid RobotsMasked Humanoid Controller (MHC): one RL-trained LSTM policy for the Digit V3 humanoid that follows masked motion directives, so joystick walking, partial-body imitation and full mimicry share one interface. Trained in MuJoCo with a three-stage curriculum and domain randomization. Author-reported: 0.450% simulated failure rate and 5% on hardware over 20 motions, with no code released.Paper · 2024-07-21Multi-Agent Causal Discovery Using Large Language ModelsMAC runs causal discovery as two LLM debate stages. A debate picks a statistical algorithm (PC, FCI, GES and others) that runs on the data to give a first graph. Meta Fusion turns that graph into text, and an Affirmative-Negative-Judge debate refines it with metadata. Author-reported: best mean rank over five benchmarks; first on 10 of 15 points with Gemini-2.0-Flash.Paper · 2024-07-19Words2Contact: Identifying Support Contacts from Verbal Instructions Using Foundation ModelsWords2Contact maps a spoken instruction to a support-contact pixel on a humanoid's camera image. An LLM writes arithmetic over GroundingDINO boxes (or CLIPSeg segments the object), and users correct the point until they confirm. Talos succeeded in all four reaching tasks. Best LLM+VLM combination: median overall success 0.61 on 78 samples (author-reported).Paper · 2024-07-18Retrieval-Augmented Generation for Natural Language Processing: A SurveySurvey of retrieval-augmented generation (RAG) for NLP. Splits RAG into a retriever, four retrieval fusions (query-based, logits-based, latent, parametric), a generator, training and knowledge-base updates, tasks, evaluation and deployment. Gives a fusion comparison table and pseudocode. Runs no experiments of its own; its quantitative claims are quoted from cited work.Paper · 2024-07-16Multi-Step Reasoning with Large Language Models, a SurveySurvey of prompt-based multi-step reasoning in LLMs. It sorts 31 approaches by how they generate steps, evaluate them and control the search over them, then lists their reported benchmark gains (author-reported by the cited papers, not re-run). Chain-of-thought, self-consistency and tree search are the core. The paper proposes no method of its own and publishes no code.Paper · 2024-07-15On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M MicrocontrollersOn-device training on Cortex-M MCUs with fully quantized training (FQT): forward and backward passes use the inference quantization scheme, scale and zero point are re-fit after each update, and per-sample error channels are pruned by a loss-dependent rate. Author-reported: 34.8% less memory and 49.0% lower latency than MCUNet on CIFAR-10, and a 6.64x average backprop speedup at lambda_min=0.1.Paper · 2024-07-13Graph Transformers: A SurveySurvey of graph transformers. Sorts how graph structure enters self-attention (node and edge positional encodings, message-passing placements, attention bias) and how attention is computed (quadratic, linear, local). Groups models into shallow, deep, scalable and pre-trained types, reviews applications, and gives a short design guide. The paper runs no experiments of its own.Paper · 2024-07-11FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisionExact attention kernel for Hopper (H100) GPUs that overlaps TMA loads, WGMMA matrix multiplies and softmax across warpgroups, with an FP8 path using block quantization. Author-reported: FP16 forward 1.5-2.0x faster than FlashAttention-2, up to 740 TFLOPs/s; FP8 near 1.2 PFLOPs/s; 2.6x lower FP8 error.Paper · 2024-07-10MambaVision: A Hybrid Mamba-Transformer Vision BackboneMambaVision is a hierarchical image backbone: two convolutional stages, then two stages that mix a redesigned Mamba (SSM) block with self-attention, with attention in the last half of each stage. The Mamba mixer adds a convolutional branch beside the SSM branch. Author-reported: 82.3% ImageNet-1K top-1 at 6298 img/s for the 31.8M-parameter Tiny model, and 84.2% for Base.Paper · 2024-07-04Differentiating Between Human-Written and AI-Generated Texts Using Automatically Extracted Linguistic FeaturesTen essays (five IELTS task samples, five GPT-4 essays matched in length) were profiled with the Open Brain AI web tool into 94 phonological, morphological, syntactic and lexical measures. Wilcoxon tests and per-domain LASSO found AI text higher in nouns, alveolar consonants and difficult words, and lower in pronouns, dentals and function words. Author-reported; five texts per group.Paper · 2024-07-01Accelerate Intermittent Deep InferenceAdapts intermittent-aware neural architecture search (iNAS) to microcontrollers with about 2 KB of SRAM. It shrinks a large MCUNet teacher by distillation and pruning, then searches under power-cycle budgets. Author-reported: +19.1% accuracy (CIFAR-10) and +12.0% (Tiny ImageNet) over HW-NAS, 1.33x and 1.27x peak memory, and 12% more latency than iNAS on CIFAR-10. No official code.Paper · 2024-07-01Diffusion Forcing: Next-token Prediction Meets Full-Sequence DiffusionDiffusion Forcing trains a causal sequence model where every token has its own independent noise level. Sampling with a per-token noise schedule gives stable long video rollouts, Monte Carlo Guidance over future tokens and flexible-horizon planning. Author-reported: 141.7 average reward on single-task Maze2D (Diffuser* 119.5) and CRPS_sum at or near the best on several time-series datasets.Paper · 2024-06-27ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual DataManiWAV records contact sound with a microphone embedded in a handheld UMI gripper, so demonstrations can be collected in the wild. A diffusion policy fuses CLIP ViT vision features and AST audio features with a transformer, and audio noise augmentation narrows the gap to robot deployment. Author-reported: 85% vs 40% (vision only) on whiteboard wiping and 90% vs 25% on bagel flipping.Paper · 2024-06-24Reducing the Memory Footprint of 3D Gaussian SplattingShrinks a 3D Gaussian Splatting scene about 27x on disk and renders about 1.7x faster. It prunes redundant Gaussians by a resolution-aware neighbour count, drops unneeded spherical-harmonic bands per Gaussian, and stores the rest in k-means codebooks with half-float positions. Author-reported PSNR stays within about 0.3 dB of 3DGS.Paper · 2024-06-21LiCS: Navigation using Learned-imitation on Cluttered SpaceLiCS trains a small Transformer by behaviour cloning on the commands of an expert local planner (FSMT or a joystick driver) in simulation. Gaussian noise on executed actions widens the recorded states. A geometric safety check gates motion on hardware. Author-reported: 100% success on 66 benchmark worlds at 1.4 m/s (FSMT 99.49%), and 91.67% vs 62.50% on 16 hard worlds.Paper · 2024-06-20Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract TheoryNine gpt-3.5-turbo agents with sampled traits, a 30-action memory and farm, rob, trade or donate choices share a food-and-land world. Robbery is a sigmoid-of-strength contest, and conceding creates superior-subordinate contracts. Author-reported: across 85 runs, the robbery rate falls from 0.391 to 0.037 after a sovereign forms, and the farm rate rises from 0.477 to 0.737.Paper · 2024-06-19ModSec-Learn: Boosting ModSecurity with Machine LearningModSec-Learn trains a linear SVM, logistic regression or random forest on which OWASP CRS SQL-injection rules fire in ModSecurity, so learned weights replace the fixed severity scores. Author-reported, on a balanced SQLi test set: TPR at 1% FPR is 99.02-99.45% at paranoia level 4, against 68.55% for vanilla ModSecurity. L1 regularization sets 18 rules to zero weight.Paper · 2024-06-18AgentReview: Exploring Peer Review Dynamics with LLM AgentsAgentReview simulates ICLR-style peer review with LLM agents: three reviewers with written traits, an author who rebuts, and an area chair who meta-reviews and accepts 32% of papers. One trait is varied at a time against a neutral baseline. With one malicious reviewer, 37.1% of decisions differ from baseline (author-reported). Official code is Apache-2.0.Paper · 2024-06-18ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All ToolsTechnical report for the ChatGLM family, centred on GLM-4: pre-training on about ten trillion tokens, grouped-query attention, RMSNorm and SwiGLU, SFT plus RLHF, and GLM-4 All Tools, which picks a browser, Python, image or user-defined tool itself. Author-reported: GLM-4 (0520) scores 83.3 MMLU, 93.3 GSM8K and 78.5 HumanEval.Paper · 2024-06-17A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large DatasetsBuilds a hierarchy of 3D Gaussians for very large street-level scenes: merged interior nodes, cuts chosen by projected size, and smooth level switching. Chunks are trained independently, then consolidated. Author-reported: real-time rendering of multi-km scenes, and 26.62 dB PSNR on one SmallCity chunk against 25.34 dB for 3DGS.Paper · 2024-06-13OpenVLA: An Open-Source Vision-Language-Action ModelOpenVLA is a 7B-parameter vision-language-action model: a Prismatic VLM with fused DINOv2+SigLIP vision and a Llama 2 7B backbone, fine-tuned to output 7 robot-action dimensions as 256-bin tokens on 970k Open X-Embodiment episodes. Authors report 70.6% (BridgeData V2) and 85.0% (Google robot) mean success, above RT-2-X (55B), and LoRA fine-tuning that matches full fine-tuning.Paper · 2024-06-12YOLO advances to its genesis: a decadal and comprehensive review of the You Only Look Once (YOLO) seriesA review of the YOLO object detectors from YOLOv1 to YOLOv12, plus YOLO-NAS, YOLO-X, YOLO-R, DAMO-YOLO and Gold-YOLO. It walks the versions in reverse order and lists each one's limits, five application areas and outlook. It runs no experiments, so all numbers come from the cited works.Paper · 2024-06-11Simple and Effective Masked Diffusion Language ModelsMDLM trains an encoder-only transformer to fill in masked tokens, under a continuous-time, Rao-Blackwellized ELBO that is a weighted mix of masked-LM losses. Sampling starts from all masks and unmasks; a semi-autoregressive variant generates arbitrary lengths. Author-reported: LM1B perplexity bound 23.00 at 327B tokens vs 20.86 for a retrained AR model; OWT 23.21 vs 17.54 for AR.Paper · 2024-06-03Learning-based legged locomotion; state of the art and future perspectivesSurvey of learning-based legged locomotion, mostly quadrupeds, with a shorter part on bipeds. Covers RL setup, sensing and reward design, curriculum, hierarchical and privileged (teacher-student) training, sim-to-real methods, hybrids with model-based control, and open problems. Contains no new method or experiments; figures quoted are from cited works.Paper · 2024-06-03MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding BenchmarkMMLU-Pro is a 12,032-question, 14-subject multiple-choice benchmark that extends MMLU from four to ten answer options, adds reasoning-heavy questions and removes trivial or noisy ones. Scores drop 16-33% versus MMLU and vary less across 24 prompt styles. Evaluated with 5-shot chain of thought; GPT-4o scores 72.6%.Paper · 2024-06-03PDP: Physics-Based Character Animation via Diffusion PolicyPDP trains a diffusion policy for simulated humanoids by behaviour cloning on data from RL experts. The experts act with action noise, but only their clean actions are stored, so the policy learns corrections from off-track states. Author-reported: 100% in-distribution and 96.3% out-of-distribution perturbation recovery on Bump'em, 97.1% success on AMASS test, and 57.1% on text prompts.Paper · 2024-06-03SpatialRGPT: Grounded Spatial Reasoning in Vision Language ModelsSpatialRGPT adds region prompts (boxes or masks) and an optional relative-depth plugin to a LLaVA-style VLM. It is trained on OSD, 1M OpenImages images with 3D scene-graph QA built from detection, metric depth and calibration. Author-reported: 91.8 qualitative accuracy on SpatialRGPT-Bench (GPT-4V 58.1) and 82.3 on BLINK relative depth (7B).Paper · 2024-05-31Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space DualityA state-space model with one scalar decay per head and masked linear attention are dual forms of one matrix family. Derives SSD, a chunked algorithm that mixes quadratic blocks with a linear state recurrence, and the Mamba-2 block built on it. Author-reported: SSD is 2-8x faster than Mamba's scan at state size 64; Mamba-2 2.7B averages 60.2 zero-shot vs 59.9 for Mamba-2.8B.Paper · 2024-05-28Hierarchical World Models as Visual Whole-Body Humanoid ControllersPuppeteer pretrains one TD-MPC2 world model to track 836 MoCap clips on a simulated 56-DoF humanoid, then trains a visual high-level TD-MPC2 planner that outputs end-effector commands to the frozen tracker. It matches TD-MPC2 return on 8 tasks except stairs, where TD-MPC2 scores higher, and a 51-person study preferred its motions (author-reported).Paper · 2024-05-28The Widening Gap: The Benefits and Harms of Generative AI for Novice ProgrammersLab study of 21 CS1 students solving one C++ problem with GitHub Copilot and ChatGPT, coded for metacognitive difficulties with think-aloud and eye tracking. Students who struggled accepted more Copilot suggestions (no test reported) and often reached fixes through ChatGPT. Replicates Prather et al. (2018) with GenAI added. Small sample, qualitative claims, no code.Paper · 2024-05-23TimeMixer: Decomposable Multiscale Mixing for Time Series ForecastingTimeMixer is an MLP-only forecaster. It average-pools the input into several scales, splits each scale into seasonal and trend parts, mixes seasonal parts bottom-up and trend parts top-down, then sums one linear predictor per scale. Author-reported: lowest MSE on all 8 long-term benchmarks (input 96) and best scores on PEMS and M4.Paper · 2024-05-23TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving ScenesTopoLogic predicts lane connections at intersections from two matrices: a learnable function of the gap between lane endpoints, and a sigmoid of lane-query similarity. They are fused with learnable weights and fed to a GNN in a lane decoder. On OpenLane-V2 subset_A (v2.1.0) it reports TOP_ll 23.9 and OLS 44.1, against 10.9 and 39.8 for TopoNet (author-reported).Paper · 2024-05-23YOLOv10: Real-Time End-to-End Object DetectionYOLOv10 removes non-maximum suppression from YOLO detectors. It trains a one-to-many and a one-to-one head with a shared matching metric, and only the one-to-one head runs at inference. Architecture changes cut cost in several components. Author-reported COCO AP ranges from 38.5 (N) to 54.4 (X), at 1.84 to 10.70 ms on a T4 with TensorRT FP16.Paper · 2024-05-20Octo: An Open-Source Generalist Robot PolicyOcto is a transformer robot policy pretrained on about 800k Open X-Embodiment trajectories. It takes camera views and a language or goal-image task, and a diffusion head decodes action chunks. New sensors or action spaces are added at finetuning time. Author-reported: 72% average success over six finetuning setups versus 20% for training from scratch, the best baseline.Paper · 2024-05-19Novel Interpretable and Robust Web-based AI Platform for Phishing Email DetectionClassic spam-filter pipeline for emails: six public spam datasets merged into 82,486 emails, TF-IDF or Word2Vec features, then linear SVC, multinomial Naive Bayes and random forest. Linear SVC on TF-IDF is the best reported model, with LIME word explanations and a Flask web app. Author-reported: accuracy 0.991, F1 0.99 on one 80/20 split.Paper · 2024-05-08Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous DrivingSemi-supervised LiDAR semantic segmentation. LaserMix cuts two scans into bands by laser-beam angle and swaps bands; a teacher-student pair trains on the mixed scans. LaserMix++ adds camera-to-LiDAR feature distillation, multi-modal mixing and text-aligned guidance. Author-reported: Cylinder3D on SemanticKITTI with 1% labels reaches 56.2 mIoU, against 45.4 supervised-only and 50.6 for LaserMix.Paper · 2024-05-07xLSTM: Extended Long Short-Term MemoryxLSTM adds exponential gating with a normalizer and stabilizer to the LSTM, plus sLSTM (scalar memory, per-head memory mixing) and mLSTM (matrix memory with a covariance update and no hidden-to-hidden links, so it runs in parallel). Residual stacks reach 13.43 validation perplexity on 15B SlimPajama tokens, below Llama (14.25) and Mamba (13.70) (author-reported).Paper · 2024-04-30KAN: Kolmogorov-Arnold NetworksKolmogorov-Arnold Networks (KANs) replace an MLP's linear weights and fixed node activations with learnable univariate B-spline functions on edges; nodes only sum. Grids can be refined without retraining, and pruning yields small, readable formulas. Authors report higher accuracy than MLPs with far fewer parameters on small function-fitting, PDE and science tasks.Paper · 2024-04-29Capabilities of Gemini Models in MedicineMed-Gemini: Gemini 1.0 and 1.5 models tuned for medicine, with web search learned by self-training plus an uncertainty-guided search at inference, multimodal fine-tuning, and long-context EHR and video tasks. Author-reported: 91.1% on MedQA (USMLE), state of the art on 10 of 14 benchmarks. Models and model code are not released.Paper · 2024-04-26MaPa: Text-driven Photorealistic Material Painting for 3D ShapesMaPa paints procedural material graphs onto a 3D mesh from a text prompt. It splits the mesh into segments, has a segment-conditioned ControlNet render a view, groups segments by material class and colour, retrieves a graph per group and fits it by differentiable rendering. Author-reported: FID 88.3 and 87.3 on chairs and ABO, against 94.6 and 109.8 for the best baselines.Paper · 2024-04-24From Local to Global: A Graph RAG Approach to Query-Focused SummarizationGraphRAG has an LLM extract entities and relations into a knowledge graph, clusters it hierarchically with Leiden, and pre-writes LLM summaries per community. Queries are answered map-reduce style over those summaries. Author-reported: beats vector RAG on comprehensiveness (72-83% win rate) and diversity (62-82%) over ~1M-token corpora.Paper · 2024-04-22Phi-3 Technical Report: A Highly Capable Language Model Locally on Your PhonePhi-3-mini: a 3.8B decoder-only model trained on 3.3T tokens of filtered web and synthetic data, then SFT and DPO. Quantized to 4 bits it runs on an iPhone 14 at over 12 tokens/s. Author-reported MMLU 68.8 (5-shot) and MT-bench 8.38; 7B and 14B siblings, MoE and vision variants follow.Paper · 2024-04-21A Survey on the Memory Mechanism of Large Language Model based AgentsSurvey of memory in LLM-based agents. Defines memory as write, manage and read operations over three sources (inside-trial, cross-trial, external knowledge), in textual or parametric form. Reviews evaluation (direct and indirect), applications and open problems. No experiments; its tables classify 28 systems by source, form and operation.Paper · 2024-04-19PhysDreamer: Physics-Based Interaction with 3D Objects via Video GenerationEstimates a per-particle Young's modulus for a static 3D Gaussian object. A video model proposes how the object moves, and MPM simulation with differentiable rendering is fitted to that motion. The fitted material then responds to new forces. Author-reported: 80.8% of 2AFC judgements prefer it over PhysGaussian on motion realism; no physical ground truth.Paper · 2024-04-16VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real TimeVASA-1 turns one portrait and a speech clip into a talking-face video. A 3D-aided encoder learns a disentangled face latent space; a diffusion transformer generates head-pose and facial-dynamics latents from audio, in sliding windows, with optional gaze, distance and emotion controls. Author-reported: 512x512 video at up to 40 FPS online, best lip sync and FVD against three baselines.Paper · 2024-04-15nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image SegmentationBenchmarks CNN, Transformer and Mamba 3D segmentation models on six datasets with five-fold CV on nnU-Net splits. Recent claims mostly fail against the original nnU-Net; CNN U-Nets (ResEnc, MedNeXt, STU-Net) lead. BTCV and BraTS separate methods poorly. Author-reported.Paper · 2024-04-14Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random FieldsTrains a character-level neural CRF for named entity recognition on a low-resource language jointly with high-resource related languages. Language-specific word embeddings and a language vector share a character LSTM, BiLSTM and tag layers. Up to 9.8 F1 over a log-linear CRF baseline with 100 target sentences.Paper · 2024-04-12A Review of Reward Functions for Reinforcement Learning in the context of Autonomous DrivingReview of the reward functions used to train RL driving agents. Splits the reward terms into safety, progress, comfort and traffic-rule categories, and finds summed or hand-weighted aggregation, no context awareness and no validation method. Proposes rulebooks, reward machines and a reward validation framework. No experiments.Paper · 2024-04-09Autonomous Driving Small-Scale Cars: A Survey of Recent DevelopmentSurvey of small-scale autonomous-car platforms, from 1/5-scale racers to swarm robots: hardware, software, simulators, sensors, and the driving tasks run on them in modular and end-to-end pipelines. Tables are author-compiled; no experiments; the proposed baseline pipeline is not evaluated.Paper · 2024-04-03Empowering Biomedical Discovery with AI AgentsA perspective, not an experiment. It defines four levels of autonomy (0 to 3) for biomedical AI agents, scored on hypothesis, experiment and reasoning, and a module roadmap of perception, interaction, memory and reasoning. It says Level 2 and 3 agents have not yet been realised and lists open challenges in reliability, evaluation, data and governance. No results are reported.Paper · 2024-04-03Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale PredictionVAR replaces raster-order next-token prediction with next-scale prediction: a multi-scale VQ tokenizer turns an image into token maps from 1x1 to 16x16, and a GPT-style transformer predicts each map from the coarser ones in one pass. Author-reported FID 1.73 on ImageNet 256 for the 2.0B model with rejection sampling.Paper · 2024-03-31End-to-End Autonomous Driving through V2X CooperationUniV2X is an end-to-end planner for vehicle-infrastructure cooperation from cameras. The roadside sends sparse agent and lane queries plus one dense occupancy map and its flow; the vehicle matches and fuses them before planning. Author-reported on DAIR-V2X: average collision rate 0.34% vs 0.89% without fusion, at 8.09e5 BPS vs 8.19e7 for dense BEV fusion.Paper · 2024-03-27Learning in PINNs: Phase transition, diffusion equilibrium, and generalizationTracks full-batch Adam on PINNs with a batch-wise gradient SNR. Finds a third phase, "diffusion equilibrium", after fitting and diffusion, where sample gradients align and test error drops. Proposes residual-based attention (RBA): a moving average of normalised residuals that reweights collocation points. Author-reported: Allen-Cahn reaches 10% relative L2 error in 10k iterations, vanilla in 100k.Paper · 2024-03-262D Gaussian Splatting for Geometrically Accurate Radiance Fields2DGS replaces the 3D Gaussian blobs of 3D Gaussian Splatting with flat, oriented 2D Gaussian disks, rendered by exact ray-splat intersection and trained with depth-distortion and normal-consistency losses. Meshes come from TSDF fusion of rendered depth. Author-reported DTU Chamfer distance is 0.80 vs 1.96 for 3DGS, with about 11 minutes of training per scene.Paper · 2024-03-24Robust-Locomotion-by-Logic: Perturbation-Resilient Bipedal Locomotion via Signal Temporal Logic Guided Model Predictive ControlPlans push recovery for the Cassie biped with an MPC whose cost maximises the robustness degree of a signal temporal logic spec: the apex must re-enter a Riemannian safety region within two steps. Smooth min/max operators make it an NLP; learned leg-collision MLPs allow crossed-leg steps. Author-reported: larger tolerated impulses than ALIP-MPC and an LTL planner in simulation.Paper · 2024-03-22Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal EstimationPredicts metric depth and surface normals from one RGB image, given the focal length. Training images are mapped to one canonical focal length, and a ConvGRU block refines depth and normal jointly, with a depth-normal consistency loss so normals can learn from depth-only data. Author-reported: NYUv2 zero-shot AbsRel 0.067 (ViT-g), KITTI zero-shot AbsRel 0.051 (ViT-g).Paper · 2024-03-21AI and Memory WallAnalysis, not a new method: peak hardware FLOPS grew 3.0x per two years against 1.6x for DRAM and 1.4x for interconnect bandwidth, so memory bandwidth becomes the bottleneck. Decoder Transformers at low batch size have far lower arithmetic intensity than encoders, so they run slower despite similar FLOPs. The paper lists directions for training, quantization, pruning and accelerator design.Paper · 2024-03-21MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View ImagesPredicts 3D Gaussians from two posed views in one forward pass. A plane-sweep cost volume over 128 inverse-depth planes, refined by a U-Net, gives per-pixel depth, unprojected to Gaussian centers. Trained with photometric loss only. Author-reported: PSNR 26.39 on RealEstate10K at 0.044 s, with 12.0M parameters versus 125.4M for pixelSplat.Paper · 2024-03-21The Opportunities and Risks of Large Language Models in Mental HealthNarrative review of LLMs for mental health education, assessment and intervention. It covers opportunities (Psy-LLM, ChatGPT and Med-PaLM 2 evaluations, chatbots) and risks (inequity, unethical practice, reliability, inaccuracy, transparency, lack of human involvement), with a risk-by-use table and recommendations. It runs no new experiments; quoted numbers come from cited studies.Paper · 2024-03-19DROID: A Large-Scale In-The-Wild Robot Manipulation DatasetDROID is a robot manipulation dataset of 76k successful Franka demonstrations (350 h) across 564 scenes, collected by 50 people over 12 months. Co-training a diffusion policy on a 50/50 mix with it beats no co-training and Open X-Embodiment co-training; the author-reported gains are 22% absolute in-distribution and 17% out-of-distribution.Paper · 2024-03-18SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video DiffusionSV3D fine-tunes Stable Video Diffusion to render a 21-frame orbit around an object from one image, optionally steered by camera elevation and azimuth. A two-stage 3D pipeline adds illumination modelling and masked score distillation. Author-reported on GSO: novel-view PSNR 21.26 against 18.34 for the best baseline, and Chamfer distance 0.024 against 0.039.Paper · 2024-03-17Customized large language models can outperform Community Notes in correcting misinformationMUSE asks an LLM (GPT-4 in the main setup) for search queries, retrieves web pages from publishers ranked by factuality and bias, extracts refuting and contextual evidence, and writes a reply with source links. Image posts add captions, celebrity labels and OCR. Author-reported: expert-rated overall quality 8.1/10 vs 6.3 for high-helpfulness Community Notes.Paper · 2024-03-14LocalMamba: Visual State Space Model with Windowed Selective ScanLocalMamba reorders image tokens so small windows are scanned together, runs selective scans in four directions per block with channel attention, and searches per layer which of eight scan directions to keep. Author-reported: LocalVim-T reaches 76.2% ImageNet top-1 at 1.5G FLOPs, 3.1 points above Vim-Ti.Paper · 2024-03-13Gemma: Open Models Based on Gemini Research and TechnologyGemma: open 2B and 7B decoder-only Transformers trained on 3T and 6T tokens with Gemini-derived architecture and recipes, released as pretrained and instruction-tuned checkpoints. Author-reported: the 7B model averages 56.9 on 18 academic benchmarks vs 54.5 for Mistral 7B.Paper · 2024-03-12Learning Generalizable Feature Fields for Mobile ManipulationGeFF is a scene-level generalizable neural feature field. One feed-forward pass over posed RGB-D frames gives SDF, point-cloud and CLIP-aligned features (distilled from MaskCLIP) for navigation and grasping. Author-reported on a quadruped with an arm: 62.4% overall success vs 43.2% for LERF at 0.39 s latency. No official code.Paper · 2024-03-11An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language ModelsFastV ranks the image tokens of a vision-language model by attention after an early decoder layer and drops the lower-ranked ones from deeper layers. No training. With K=2 and R=50%, LLaVA-1.5-13B keeps its average score on four benchmarks (73.6) at about 55% of the theoretical FLOPs, author-reported.Paper · 2024-03-11VideoMamba: State Space Model for Efficient Video UnderstandingRuns bidirectional Mamba state-space blocks over flattened video patches in a plain ViT-style stack, so cost grows linearly with clip length. A smaller trained copy guides the larger model (self-distillation). Author-reported: 84.0% ImageNet-1K top-1 at 576px, 83.3% Kinetics-400 at 64x384px, 68.4% Something-Something V2 at 16x288px.Paper · 2024-03-08Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextGemini 1.5 Pro (a sparse mixture-of-experts Transformer) and Flash (a dense model distilled from Pro) take text, video, audio and code in one context of up to millions of tokens. Author-reported: near-perfect needle recall to 1M tokens and 99.2% at 10M text tokens; Pro matches or beats Gemini 1.0 Ultra on most core benchmarks. Architecture and training details are largely not stated.Paper · 2024-03-07A Survey on Human-AI Collaboration with Large Foundation ModelsSurvey of human-AI collaboration with large foundation models. It organises the field into build-time human guidance (data curation, preference-based optimisation, evaluation), five run-time collaboration patterns, ethics and governance, six application domains and open challenges. It reports no experiments of its own; the figures it quotes come from cited studies.Paper · 2024-03-07Machine learning and information theory concepts towards an AI MathematicianPosition essay, no experiments. Proposes that an AI mathematician should favour theorems that compress the set of provable statements: short to describe and few derivation steps from many provable statements. Suggests surprise-based interestingness, GFlowNet-style conjecture sampling and goal-conditioned RL with lemmas as subgoals. No method is evaluated.Paper · 2024-02-28A Survey on Recent Advances in LLM-Based Multi-turn Dialogue SystemsSurvey of LLM-based multi-turn dialogue. Covers LLM architectures and adaptation (full fine-tuning, adapters, LoRA, instruction tuning, prompting, RAG), task-oriented and open-domain systems, evaluation metrics, datasets and open problems. Runs no experiments and reports no new method results.Paper · 2024-02-27EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak ConditionsEMO turns one portrait and an audio clip into a talking or singing video with a diffusion model: Stable Diffusion 1.5 plus temporal layers, a ReferenceNet for identity, audio attention, and weak face-box and head-speed conditions. Author-reported on HDTF: best FID (8.76) and FVD (67.66); SyncNet (3.89) trails Wav2Lip and SadTalker.Paper · 2024-02-27LLM-Powered Socially Assistive Robot-Delivered Cognitive Behavioral Therapy Exercises: an Exploratory Study with University StudentsA 15-day exploratory study (38 university students) of CBT homework delivered by a GPT-3.5 socially assistive robot, a text chatbot using the same LLM, or paper worksheets. Distress (OQ-45) fell in the robot and worksheet groups but not the chatbot group. Robot and chatbot sessions ended with lower anxiety than worksheet sessions in a mixed-effects model.Paper · 2024-02-23MambaIR: A Simple Baseline for Image Restoration with State-Space ModelImage restoration backbone that replaces window attention with a four-direction Mamba scan (VSSM), adding a conv bottleneck with channel attention and learnable skip scales in each block. Author-reported: beats SwinIR on classic SR (e.g. +0.41 dB Manga109 x2), lightweight SR and Gaussian color denoising (+0.48 dB Urban100, sigma 50); close to Restormer on real denoising.Paper · 2024-02-21Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation DialoguesZero-shot evaluation of GPT-3.5, GPT-4, Mistral-7B, Vicuna-13B/33B and Wizard-13B on 35 tasks built from four bilateral negotiation dialogue datasets, against a majority baseline and a fine-tuned Flan-T5. GPT-4 leads most tasks; models agree poorly with humans on subjective judgements; generated replies are near human on coherence but not on strategy. Author-reported.Paper · 2024-02-21KorNAT: LLM Alignment Benchmark for Korean Social Values and Common KnowledgeKorNAT is a multiple-choice benchmark for how well an LLM matches South Korea. Social-value items score a choice by the share of a 6,174-person survey that picked it; common-knowledge items come from Korean school curricula and score accuracy against a 0.6 reference. Of seven LLMs tested, three reach 0.6 on common knowledge (author-reported).Paper · 2024-02-20Investigating Cultural Alignment of Large Language ModelsSimulates the World Values Survey in Egypt and the US: LLMs answer as real respondents' demographic personas, scored against the survey. Models match the US survey more than the Egypt one, and better when prompted in that country's dominant language. Anthropological Prompting, which asks for framework-guided reasoning first, lifts GPT-3.5's Egypt soft score from 0.483 to 0.510. Author-reported.Paper · 2024-02-19Feudal Networks for Visual NavigationFeudalNav reaches image goals without RL, graphs, odometry or a metric map. A self-supervised latent memory map feeds a ResNet-18 waypoint predictor trained on human point-clicks, and a worker turns waypoints into turns and steps. On Gibson, curved-path success averages 40.50 for Stacked FeudalNav vs 23.97 for NRNS (author-reported).Paper · 2024-02-15BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical DomainsBioMistral continues pre-training Mistral 7B Instruct v0.1 on 3B tokens of PubMed Central open-access text (commercial-use subset), then tests it on 10 English medical QA tasks, 7 machine-translated languages, AWQ and BitsandBytes quantization, and TIES/DARE/SLERP merges with the base model. Author-reported 3-shot English average: SLERP merge 55.4 vs Mistral 7B Instruct 51.2.Paper · 2024-02-15Chain-of-Thought Reasoning Without PromptingCoT-decoding branches on the top-k alternative tokens of the first decoding step, continues each branch greedily, and keeps the answer whose tokens have the widest top-1 vs top-2 probability margin, a sign of a chain of thought. No prompt or tuning. Author-reported: GSM8K 9.9% to 25.1% on Mistral-7B and 34.8% to 63.2% on PaLM-2 L.Paper · 2024-02-09Large Language Models: A SurveySurvey of large language models: the GPT, LLaMA and PaLM families, how LLMs are built (data cleaning, tokenisation, positional encodings, pre-training, alignment, decoding), how they are used and augmented (prompting, retrieval, tools, agents), datasets, metrics, and benchmark tables compiled from reported third-party numbers. Proposes no new method; no training runs of its own.Paper · 2024-02-07The Effect of Sampling Temperature on Problem Solving in Large Language ModelsMeasures how sampling temperature affects LLM accuracy on multiple-choice questions: nine models, five prompt styles, ten benchmark sets, ten answers per question. Accuracy shows no significant change from 0.0 to 1.0 (GPT-3.5, 1,000 questions: H(10)=10.439, p=0.403) and falls above 1.0. Author-reported; authors recommend 0.0.Paper · 2024-02-06Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and DefensesSurvey of model inversion (MI) attacks and defenses on DNNs: rebuilding training data from a released model. Covers more than 130 papers (2014 to 30 April 2026) on images, text and graphs, with a taxonomy by attacker access, modality and task, per-method tables of generators, search spaces and losses, and defense families. No new benchmark numbers; the figures are qualitative.Paper · 2024-02-05Effects of Shared Control on Cognitive Load and Trust in Teleoperated Trajectory TrackingA 24-person user study of shared control in teleoperated 3-D trajectory tracking on a robot arm, varying robot autonomy as a blend of robot and human input. More autonomy lowered self-reported workload (NASA-TLX) and raised self-reported trust (MDMT and a single trust item). Tapping and pupil measures showed no autonomy effect, and load and trust did not interact. Author-reported.Paper · 2024-02-05M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge DistillationOne XLM-RoBERTa-based encoder (extended to 8,192 tokens) yields dense, sparse and multi-vector scores for multilingual retrieval. Self-knowledge distillation trains the three together, with their weighted sum as teacher. Author-reported: MIRACL nDCG@10 71.5, MKQA Recall@100 75.5, MLDR nDCG@10 65.0 (averages, all three modes).Paper · 2024-02-04A Survey on Robotics with Foundation Models: toward Embodied AISurvey of foundation models (LLMs, VLMs) for robotic manipulation. It groups work into high-level planning (plan forms and the vision, knowledge and feedback that assist them) and low-level control (policy learning, environment modeling, representation learning), then covers datasets, simulators, benchmarks and open challenges. No new experiments; its numbers are dataset and task counts.Paper · 2024-02-02KTO: Model Alignment as Prospect Theoretic OptimizationKTO aligns LLMs from binary desirable/undesirable labels instead of preference pairs. Each output is scored against a reference point with a prospect-theory value function. Author-reported: it matches or beats DPO from 1B to 30B parameters, and on Zephyr-beta-SFT raises GSM8K from 40.0 (DPO) to 53.5.Paper · 2024-01-30Security and Privacy Challenges of Large Language Models: A SurveySurvey without experiments of LLM security and privacy risks, grouped as prompt injection, jailbreaking, backdoor and data-poisoning attacks, gradient leakage, membership inference and PII leakage. Lists the defences named for each family, plus application risks and open gaps. Most attacks and defences were tested on small models, not LLMs. Numbers are quoted from cited papers.Paper · 2024-01-22Revolutionizing Finance with LLMs: An Overview of Applications and InsightsA survey of LLM use in finance across trading, forecasting, risk management and financial QA, plus a small evaluation of GPT-4 on six datasets with zero-shot or one-shot prompts. Reported scores: 0.78 FPB accuracy, 0.79 FiQA-SA, 0.81 NER entity F1, 0.64 FinQA, 0.73 ConvFinQA, 0.53 stock movement. No baseline; author-reported.Paper · 2024-01-20Exploring consumers response to text-based chatbots in e-commerce: The moderating role of task complexity and chatbot disclosureSurvey of 299 e-commerce consumers analysed with OLS regression. Perceived empathy and friendliness of a text-based chatbot raise trust in it. Task complexity weakens the friendliness effect, and disclosure of the bot's identity moderates both effects. Trust raises reliance (beta 0.783) and lowers resistance (beta -0.382). Cross-sectional self-report, no experiment.Paper · 2024-01-19Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding HeadsMedusa adds K extra one-layer decoding heads on the LLM's last hidden state to predict tokens t+2..t+K+1, builds a sparse tree of candidates from their top-k outputs, verifies all in one forward pass with a tree attention mask, and accepts the longest valid prefix. No draft model. Batch size 1: 2.18-2.33x (Medusa-1, frozen backbone), about 2.8x (Medusa-2) on Vicuna.Paper · 2024-01-19Towards Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection from Force DataClassifies which glovebox task a teleoperator is doing, and forecasts the next 5 s, from a robot arm's joint positions, velocities and forces alone. A CNN classifies six tasks; a CNN+LSTM forecasts 500 frames. Reported F1: 89% for task classification, 86% for forecast intent over 5 s (author-reported).Paper · 2024-01-16The Faiss libraryFaiss is a C++ and Python toolkit for approximate nearest-neighbour search over embedding vectors. It chains a codec (product, additive, scalar or binary) with a non-exhaustive step (inverted file or graph) and optional re-ranking. Author-reported: a 1.5-trillion-vector index with 54-byte codes at about 1 s per query, and a Deep10M benchmark faster than SCANN.Paper · 2024-01-12Large Language Models Can Learn Temporal ReasoningTG-LLM fine-tunes Llama-2 13B with LoRA in two steps: it first translates a story into a temporal graph (a chronological timeline), then reasons over that graph with chain-of-thought chosen by contrastive bootstrapping and augmented with graph perturbations. Trained on TGQA, a synthetic dataset. Author-reported: on TGQA, EM 0.797 for Llama2-13B (SFT-TGR) against 0.821 for GPT-4 with CoT.Paper · 2024-01-11Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMsFinds image pairs that CLIP treats as near-identical but DINOv2 separates, builds the MMVP benchmark of simple visual questions on them, and groups failures into nine patterns. Seven of the nine patterns stay unsolved by large CLIP models; GPT-4V scores 38.7%, humans 95.7%. Interleaving CLIP and DINOv2 tokens (Interleaved-MoF) raises LLaVA's MMVP score from 5.5 to 16.7 at 224 px.Paper · 2024-01-08A Survey on 3D Gaussian SplattingSurvey of 3D Gaussian splatting (3DGS): a scene is learnable 3D Gaussians drawn by projection and alpha blending, fitted to photos with a photometric loss and density control. It compiles author-reported benchmarks, e.g. D-3DGS gains 6.83 dB PSNR over FFDNeRF on D-NeRF, and SplaTAM reaches 0.36 cm mean ATE on Replica.Paper · 2024-01-08Mixtral of ExpertsMixtral 8x7B is a decoder-only Transformer whose feed-forward blocks are 8 SwiGLU experts. A linear router picks 2 experts per token per layer, so 47B parameters are stored and 13B are active per token. Trained with a 32k context, it matches or beats Llama 2 70B and GPT-3.5 on most author-run benchmarks. The instruct model is SFT and DPO tuned (author-reported).Paper · 2024-01-02A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language ModelsSurvey of over 32 techniques for reducing hallucination in LLMs. Sorts them into prompt engineering (retrieval, self-refinement, prompt tuning) and model development (decoding, knowledge graphs, faithfulness losses, fine-tuning), with a taxonomy and a per-technique table. Runs no experiments; its only quoted result is 59.6% F1 for D&Q.Paper · 2024-01-02Large Legal Fictions: Profiling Legal Hallucinations in Large Language ModelsMeasures how often GPT-4, GPT-3.5, PaLM 2 and Llama 2 give false answers about US federal cases on 14 legal tasks. Truth comes from case metadata where possible; otherwise a GPT-4 judge flags self-contradiction across two temperature-1 answers (a lower bound). Author-reported: 58% (GPT-4) to 88% (Llama 2) on direct questions; false premises often accepted; poor calibration.Paper · 2023-12-31Improving Text Embeddings with Large Language ModelsGenerates synthetic embedding training data with GPT-4 and GPT-3.5 (about 500k examples, 93 languages), then fine-tunes Mistral-7B with LoRA (rank 16) and InfoNCE (temperature 0.02) for one epoch. Queries get a task instruction. Author-reported: MTEB average 66.6 with full data (BGE-large 64.2), 63.1 with synthetic data only; BEIR 56.9.Paper · 2023-12-31MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series ForecastingMSGNet forecasts multivariate series. It finds the k strongest periods with an FFT, reshapes the series per period, learns a separate adaptive graph and Mixhop convolution per scale, applies self-attention within each period, and sums the scales by amplitude softmax. Author-reported: average rank 1.813 in the main table, against 2.750 for TimesNet.Paper · 2023-12-24Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense RetrievalAdapts LLaMA-2-7B for dense retrieval without labels. A joint prompt asks the model to rebuild a Wikipedia passage (EBAE) and to predict the next passage (EBAR), read from two special-token embeddings; a custom attention mask keeps the two prompts apart. After LoRA fine-tuning on MS MARCO it reports 43.1 MRR@10 on passages and 56.4 average NDCG@10 on zero-shot BEIR. Author-reported.Paper · 2023-12-24ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic ManipulationFine-tunes LLaMA-Adapter (CLIP, LLaMA-7B) on simulated articulated objects to output a contact pixel and two gripper directions, using affordance and masked-pose tasks. A force-probing impedance loop and per-trial test-time adaptation follow. Author-reported: 0.54 initial-movement success on 10 unseen simulated categories vs 0.41 for the best baseline.Paper · 2023-12-24The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference PerspectiveWhen LLMs play blinded subjects in a price experiment, the price also shifts unstated covariates, which confounds the simulated demand curve. Adding covariates to the prompt can cause focalism. Telling the model the randomization design (unblinding) lowers error against a human survey across GPT models, including after fine-tuning. All results are author-reported.Paper · 2023-12-22A Survey of Reinforcement Learning from Human FeedbackSurvey of reinforcement learning from human feedback (RLHF). It covers feedback types, label collection, reward models fit by a Bradley-Terry likelihood, policy learning by RL or direct methods such as DPO, theory, benchmarks and LLM fine-tuning. Survey only: no experiments and no new results.Paper · 2023-12-20Generative Multimodal Models are In-Context LearnersEmu2 is a 37B-parameter generative multimodal model. Images become 64 embeddings interleaved with text, and one autoregressive LLaMA-based model predicts the next text token or image embedding. Scale gives few-shot in-context learning, e.g. 68.8 VQAv2 at 16 shots (author-reported).Paper · 2023-12-20Splatter Image: Ultra-Fast Single-View 3D ReconstructionPredicts one 3D Gaussian per pixel from a single image with a 2D U-Net, then renders them with Gaussian Splatting, so reconstruction is a single feed-forward pass. Trained on rendered target views with L2 plus LPIPS. Author-reported: 24.00 PSNR on ShapeNet cars (1 view, relative pose) and 21.06 PSNR on Google Scanned Objects vs 18.06 for OpenLRM.Paper · 2023-12-19Gemini: A Family of Highly Capable Multimodal ModelsTechnical report for Gemini 1.0 (Ultra, Pro, Nano): decoder-only Transformers with 32K context and multi-query attention, trained jointly on text, image, audio and video, then post-trained with SFT, reward models and RLHF. Gemini Ultra reports 90.04% on MMLU via uncertainty-routed chain-of-thought (32 samples) and state of the art on 30 of 32 benchmarks.Paper · 2023-12-18GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion ModelsGraspLDM generates 6-DoF grasp poses from object point clouds with a conditional VAE and a DDPM prior in its latent space, so a new task only retrains a small denoiser. Trained in simulation on ACRONYM, it reaches a 78% median simulated success on 63 categories and 80% and 78.75% real-robot success in two setups (author-reported).Paper · 2023-12-17Bi-directional Adapter for Multi-modal TrackingBAT adds a small bi-directional adapter to a frozen dual-stream ViT tracker so that the RGB and thermal streams prompt each other in every layer, without a fixed dominant modality. Only 0.32M parameters are trained. Author-reported: 70.2% PR and 56.3% SR on LasHeR; 86.8% MPR and 64.1% MSR on RGBT234.Paper · 2023-12-16PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUPowerInfer runs LLM inference on one consumer GPU plus CPU. Neurons that fire often (hot) stay on the GPU, rarely firing ones (cold) are computed on the CPU, and per-layer MLP predictors skip inactive neurons. An ILP solver picks the placement. Author-reported: up to 11.69x over llama.cpp on an RTX 4090 for Falcon-40B, and 82% of an A100's token rate for OPT-30B.Paper · 2023-12-14DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous DrivingDriveMLM fine-tunes a multimodal LLM (LLaMA-7B with EVA-CLIP ViT-g/14 and a LiDAR encoder) to output one of 4 speed and 5 path decisions plus a text explanation. A standard motion planner executes those decisions in CARLA. Author-reported: driving score 76.1 on CARLA Town05 Long, against 71.4 for the Apollo baseline.Paper · 2023-12-11Harmonic Mobile ManipulationTrains one reinforcement-learning policy that drives a Stretch robot's base and arm together from two RGB cameras and arm proprioception, instead of separate navigation and manipulation stages. Author-reported: 47% average success over four simulated tasks in unseen houses, 17.6 points above ReLMoGen; real-apartment success of 60-70% on door and table trials.Paper · 2023-12-11Photorealistic Video Generation with Diffusion ModelsW.A.L.T trains one transformer diffusion model on images and videos in a shared causal-autoencoder latent space, with alternating spatial and spatiotemporal window attention and AdaLN-LoRA. Reports state-of-the-art FVD on UCF-101 and Kinetics-600 and FID 2.56 on ImageNet 256 without classifier-free guidance. A three-model cascade makes text-to-video. Author-reported.Paper · 2023-12-10Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMsCompares two ways to add facts to 7B LLMs: continued unsupervised training on Wikipedia text (fine-tuning) and retrieval-augmented generation (RAG), which prepends top-K retrieved passages to the question. Tested on MMLU subsets and 910 new current-events questions. RAG beats fine-tuning in nearly all settings; paraphrases help fine-tuning learn new facts. Author-reported. No official code.Paper · 2023-12-06A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image InpaintingPowerPaint fine-tunes Stable Diffusion with one learned prompt token per inpainting task, so one model fills context, inserts text-guided objects, removes objects via a negative prompt and fits objects to masks. Author-reported: best FID in all four object-inpainting settings (e.g. 4.38 on OpenImages bounding boxes) and FID 10.16 on outpainting.Paper · 2023-12-06Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature FieldsFeature 3DGS gives each 3D Gaussian a learned N-dimensional feature and distils 2D foundation models (LSeg, SAM) into it by splatting. A 1x1 convolution lifts a smaller rendered feature to the teacher's size. Author-reported: Replica PSNR 37.01 vs 36.13 for base 3DGS; segmentation mIoU 0.782 vs 0.636 for NeRF-DFF at 14.55 vs 5.38 FPS.Paper · 2023-12-04LLMs Accelerate Annotation for Medical Information ExtractionFew-shot prompted PaLM 2 pre-labels medication fields (name, dose, mode, frequency, duration) in i2b2 2009 discharge summaries, then experts refine the labels. Author-reported: mean human time per refined document falls from 26.67 to 11.32 minutes (-58%) on 216 test documents, with refined labels close to the fully human pipeline (vertical F1 0.907 vs 0.910).Paper · 2023-12-04PixelLM: Pixel Reasoning with Large Multimodal ModelPixelLM answers in text and emits one mask per named target. A small pixel decoder reads hidden states of learned codebook tokens at two CLIP feature scales, so no SAM model is needed. Trained on MUSE, a GPT-4V-built multi-target dataset. Reports gains over LISA on MUSE, multi-referring and refCOCO+/refCOCOg at about half the TFLOPs. Author-reported.Paper · 2023-12-04Repurposing Diffusion-Based Image Generators for Monocular Depth EstimationMarigold fine-tunes only the denoising U-Net of Stable Diffusion v2 to predict affine-invariant depth, trained on synthetic Hypersim and Virtual KITTI data. It uses annealed multi-resolution noise and median ensembling of 10 runs. Zero-shot on five real datasets; best AbsRel on NYUv2, KITTI, ETH3D and ScanNet. Author-reported.Paper · 2023-12-04SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMSplaTAM runs dense RGB-D SLAM on 3D Gaussians. Each frame it fits the camera pose by rendering the map and minimising depth and colour error with the map fixed, adds Gaussians where the map is sparse or misses geometry, and refines the map over a window of keyframes. Author-reported: 0.36 cm ATE on Replica (Point-SLAM 0.52), 1.2 cm on ScanNet++ where Point-SLAM and ORB-SLAM3 fail to track.Paper · 2023-12-04Style Aligned Image Generation via Shared AttentionStyleAligned makes a set of text-to-image outputs share one style without training: in each SDXL self-attention layer, images also attend to one reference image's keys and values, after AdaIN on queries and keys. Author-reported on 100 prompts: DINO set consistency 0.51, CLIP text alignment 0.287. DreamBooth-LoRA is more consistent (0.54) but needs per-style fine-tuning.Paper · 2023-12-01DeepCache: Accelerating Diffusion Models for FreeTraining-free speedup for pretrained U-Net diffusion samplers: a full network pass runs every N steps, and the steps between reuse cached high-level features and recompute only shallow blocks. Author-reported: 2.15x throughput on Stable Diffusion v1.5 (PLMS-50, N=5) at a 0.05 CLIP drop; 4.12x on ImageNet LDM-4-G (N=5) at FID 3.59 vs 3.37.Paper · 2023-12-01FSGS: Real-Time Few-shot View Synthesis using Gaussian SplattingFits 3D Gaussian Splatting from three posed photos. Proximity-guided Gaussian unpooling adds Gaussians between sparse neighbours, and pseudo views with a monocular-depth correlation loss curb overfitting. Author-reported: 20.31 dB PSNR on LLFF at 1/8 resolution with 458 FPS, against 19.86 dB and 0.21 FPS for SparseNeRF.Paper · 2023-12-01Gaussian Grouping: Segment and Edit Anything in 3D ScenesGaussian Grouping gives each 3D Gaussian of Gaussian Splatting a 16-dimensional identity encoding, trained from SAM masks tracked across views with a 2D cross-entropy loss and a 3D kNN consistency loss. Grouped Gaussians segment anything and support removal, inpainting and style transfer. Author-reported: LERF-Mask mIoU 69.7 on figurines (LERF 33.5); Mip-NeRF 360 PSNR 28.43 vs 28.69 for 3DGS.Paper · 2023-12-01Mamba: Linear-Time Sequence Modeling with Selective State SpacesMakes the SSM step size and input/output matrices functions of each token, so a state-space model can keep or drop each input. A fused, memory-aware parallel scan computes the resulting recurrence; blocks are stacked with no attention or MLP. Author-reported: best zero-shot results against open models of the same size, and about 5x inference throughput versus Transformers.Paper · 2023-12-01ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual PromptsDraws visual prompts (boxes, ellipses, arrows, scribbles, masks) onto the image before a LLaVA-style encoder reads it, so no region module is needed. Trained on 520k marked-image instructions plus 13k GPT-4V region examples. Author-reported: Visual7W 87.91% and PointQA 71.77% (13B), ViP-Bench 48.3% (13B, boxes).Paper · 2023-11-30Event-based Continuous Color Video Decompression from Single FramesGenerates a continuous color video from one static RGB frame and the event-camera stream that follows it, queryable at any time. A learned continuous motion basis and tri-plane event features feed a fusion network that uses RAFT-refined latent-frame flow and softmax splatting. Author-reported: 28.68 dB PSNR on their E2D2 set, 25.40 dB at skip 1 on BS-ERGB.Paper · 2023-11-30Large Language Models for Travel Behavior PredictionPredicts travel mode choice and trip purpose two ways: zero-shot prompting with structured trip data and rules, and supervised MNL, RF or NN fitted on LLM text embeddings. Author-reported: zero-shot GPT-3.5 reaches 0.586 mode-choice accuracy, above supervised models trained on 10 samples, and embeddings help in that small-sample setting. Supervised models trained on 1,000 samples do best.Paper · 2023-11-30One-step Diffusion with Distribution Matching DistillationDistribution Matching Distillation (DMD) trains a one-step image generator to match a diffusion teacher's output distribution. Its gradient is the difference of two score estimates (frozen teacher, and a fake-score model trained alongside), plus an LPIPS regression to teacher outputs. Author-reported: FID 2.62 on ImageNet 64x64; 11.49 zero-shot on COCO-30k, about 30x faster than Stable Diffusion.Paper · 2023-11-30SparseGS: Sparse View Synthesis using 3D Gaussian SplattingSparseGS trains 3D Gaussian Splatting from as few as 12 (unbounded 360-degree) or 3 (forward-facing) photos. It adds a monocular depth prior with Pearson losses, softmax-scaled depth rendering, Stable Diffusion score distillation and depth-warped views at unseen angles, then prunes floaters. Author-reported PSNR on Mip-NeRF360 with 12 views: 19.37, against 17.49 for 3DGS and 19.31 for FSGS.Paper · 2023-11-29Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous DrivingDrive-WM generates six-camera nuScenes driving videos with a latent video diffusion model conditioned on 3D layouts, text and ego actions, then ranks VAD-planned trajectories by image-based map and object rewards. Author-reported: video FID 15.8 and FVD 122.7; open-loop planning L2 0.80 m and collision 0.26%.Paper · 2023-11-29Fast, Interpretable, and Deterministic Time Series Classification With a Bag-of-Receptive-FieldsBag-of-Receptive-Fields (BORF) slides dilated, strided windows over each signal, z-normalises each window, turns segment means into 3-symbol SAX words and counts them, giving a sparse, deterministic feature set for a ridge classifier. Author-reported on 158 UCR/UEA datasets: accuracy statistically tied with ROCKET and MiniRocket (9th average rank), second-best runtime.Paper · 2023-11-29Introduction to Transformers: an NLP PerspectiveTutorial-style survey of the Transformer: encoder-decoder stack, scaled dot-product and multi-head attention, sinusoidal positional encoding, post-norm and pre-norm residual blocks, warmup learning-rate schedule and decoder caching. It then covers refinements (syntax, locality, depth, efficiency, memory) and applications. It reports no experiments and no benchmark numbers.Paper · 2023-11-29MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language ModelsMM-SafetyBench pairs 1,680 GPT-4-written harmful questions across 13 policy scenarios with three generated images each (Stable Diffusion, typography, or both). It scores 12 open MLLMs with a GPT-4 judge. Typography lifts LLaVA-1.5-7B's average attack success from 41.0% to 71.5%, and a safety prefix cuts it to 17.1% (author-reported).Paper · 2023-11-29MoMask: Generative Masked Modeling of 3D Human MotionsMoMask generates 3D human motion from text. A six-layer residual VQ-VAE turns each motion into token layers. A masked transformer fills in the base layer over a few parallel rounds, and a residual transformer fills in the finer layers. Author-reported: HumanML3D FID 0.045 against 0.141 for T2M-GPT, the best FID, R-Precision and multimodal distance on that dataset.Paper · 2023-11-28Adversarial Diffusion DistillationAdversarial Diffusion Distillation trains a 1 to 4 step student from a frozen diffusion teacher with a hinge-loss discriminator on frozen ViT features plus a squared-error score distillation term. Author-reported: 1-step ADD-M reaches COCO FID-5k 19.7 and CLIP 0.326 among SD1.5-based methods, and 4-step ADD-XL beats SDXL at 50 steps in most human comparisons.Paper · 2023-11-28CLAP: Isolating Content from Style through Contrastive Learning with Augmented PromptsCLAP trains a small residual MLP on frozen CLIP text features. A contrastive loss pulls a prompt and its style-augmented copy together, so the MLP keeps content and drops style. The MLP is then applied to CLIP image and text features. Author-reported: ViT-B/16 zero-shot average over four multi-domain datasets rises from 77.4% (CLIP) to 79.9%.Paper · 2023-11-28Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in MedicineMedprompt steers GPT-4 without fine-tuning: kNN-selected few-shot exemplars, GPT-4-written chain-of-thought rationales kept only when their answer is correct, and a majority vote over 5 calls with shuffled answer options. Author-reported: 90.2% on MedQA (90.6% with 20 exemplars and 11 calls), first place on all nine MultiMedQA datasets.Paper · 2023-11-28MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkMVBench: 20 video tasks, each made by turning a static image-task question into a temporal one, built by converting 11 public video datasets into multiple-choice QA. VideoChat2, a Vicuna-7B video MLLM trained in three stages, scores 51.1% on it (author-reported), over 15 points above VideoChat (35.5%).Paper · 2023-11-28Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive DecodingTraining-free decoding for large vision-language models. Each token is sampled from (1+α)·logits(image) − α·logits(noised image), with α=1, keeping only tokens with probability ≥ β·max (β=0.1). Author-reported gains on POPE reach +5.8 accuracy and +7.4 F1 across LLaVA-1.5, InstructBLIP and Qwen-VL.Paper · 2023-11-28Power Hungry Processing: Watts Driving the Cost of AI Deployment?Measures the energy and CO2eq of 1,000 inferences for 88 models on 10 tasks: 80 task-specific models and 8 multi-purpose ones (Flan-T5, BLOOMz), run with CodeCarbon on A100 GPUs. Generative and multi-purpose models cost far more per query than task-specific ones. Text-to-image generation averages 2.9 kWh per 1,000 inferences, text classification 0.002 kWh. Author-reported.Paper · 2023-11-27MEDITRON-70B: Scaling Medical Pretraining for Large Language ModelsMEDITRON continues Llama-2 pretraining at 7B and 70B on GAP-Replay, a 48.1B-token medical corpus of guidelines, PubMed papers and abstracts, plus 1% general replay, then finetunes per benchmark. MEDITRON-70B with self-consistency CoT averages 72.0% across five medical benchmark columns, 2.8 points above Llama-2-70B (author-reported).Paper · 2023-11-27MagicAnimate: Temporally Consistent Human Image Animation using Diffusion ModelMagicAnimate animates a reference person image along a DensePose motion sequence with a latent diffusion model. It inflates the UNet with temporal attention, adds an appearance encoder whose features join the self-attention, and averages overlapping windows for long videos. Author-reported: FVD 179.07 vs 292.80 for DisCo on TikTok.Paper · 2023-11-27SeeSR: Towards Semantics-Aware Real-World Image Super-ResolutionSeeSR upscales real photos with Stable Diffusion, guided by semantic prompts from a degradation-aware tagger (DAPE, fine-tuned from RAM). Tags enter as text, tagger embeddings through cross-attention, and the low-res latent seeds sampling. Author-reported: best FID, CLIPIQA and MUSIQ on the four test sets; lower PSNR and SSIM than the best baselines.Paper · 2023-11-24GeoChat: Grounded Large Vision-Language Model for Remote SensingGeoChat is a LLaVA-1.5-style vision-language model for remote sensing: a CLIP ViT-L/14 at 504x504, a two-layer MLP adaptor and Vicuna-v1.5 7B tuned with LoRA on 318k instruction pairs built from RS detection, VQA and scene datasets. It takes region boxes and answers with boxes via task tokens. Zero-shot scene classification: 84.43% UCMerced, 72.03% AID (author-reported).Paper · 2023-11-23Cultural Bias and Cultural Alignment of Large Language ModelsPlaces five OpenAI GPT models on the Inglehart-Welzel cultural map built from World Values Survey data. Default answers sit closest to English-speaking and Protestant European countries. Cultural prompting (answering as a person born and living in a given country) lowers GPT-4o's mean distance to that country's values from 2.42 to 1.57 and improves 71 to 81% of countries for GPT-4, 4-turbo and 4o.Paper · 2023-11-21Hand-Eye CalibrationHand-eye calibration finds the fixed transform between a camera and a robot hand from several motions. Adds a second formulation MY=M'YB on 3x4 projection matrices, skipping the intrinsic/extrinsic split, and two solvers: closed-form unit-quaternion rotation then linear translation, or joint non-linear optimization. The joint solver is the most noise-robust in simulation (author-reported).Paper · 2023-11-20Efficient Neural Networks for Tiny Machine Learning: A Comprehensive ReviewReview of efficient neural networks and TinyML deployment tools for ultra-low-power Arm Cortex-M microcontrollers: distillation, pruning, quantization, weight sharing, low-rank decomposition, and frameworks such as TFLM. Compares published model size vs accuracy on MNIST, ImageNet, VWW and speech commands. No new experiments.Paper · 2023-11-16MedAgents: Large Language Models as Collaborators for Zero-shot Medical ReasoningTraining-free, zero-shot medical multiple-choice QA. LLM role-play experts (5 question fields, 2 option fields) write analyses, a summary report is voted on and revised over up to 5 rounds until all agree, then one option letter is chosen. Author-reported: GPT-4 average 86.7% over nine datasets (MedQA 83.7%) vs 85.4% for few-shot CoT with self-consistency; GPT-3.5 72.1%.Paper · 2023-11-14One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D DiffusionOne-2-3-45++ turns one RGB image into a textured mesh in about 60 s (author-reported). It fine-tunes Stable Diffusion 2 to draw six views tiled into one image, lifts them to 3D with two sparse-volume diffusion stages conditioned on DINOv2 features projected from those views, then refines texture with a TensoRF colour field. On GSO it reports F-Score 93.6 and CLIP-Sim 81.0.Paper · 2023-11-10Florence-2: Advancing a Unified Representation for a Variety of Vision TasksFlorence-2 takes an image plus a text task prompt and writes the answer as text, with boxes and polygons as quantized location tokens. A DaViT encoder and a BART-style encoder-decoder are trained with next-token loss on FLD-5B, 5.4B annotations from a model-driven data engine. The large model (0.77B) reaches 135.6 zero-shot COCO caption CIDEr.Paper · 2023-11-10FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph PerspectiveTreats each value of a lookback window as a node of one fully connected space-time graph, and forecasts with stacked Fourier-domain graph layers whose complex d-by-d weights are shared across nodes. Author-reported: best MAE and RMSE on six benchmarks at horizon 12, but not the best MAPE on Solar and Traffic. The MIT-licensed code differs from the paper in several places.Paper · 2023-11-10Frequency-domain MLPs are More Effective Learners in Time Series ForecastingForecasts with two MLPs that run on a discrete Fourier transform of the input: one mixes channels, one mixes time steps. A feed-forward head then predicts the horizon. Author-reported: MAE 0.120 on Solar and 0.011 on Traffic at 12-step horizons, and the best MAE on most long-term benchmarks; PatchTST wins on Traffic at two horizons and on one Electricity horizon.Paper · 2023-11-09LLaVA-Plus: Learning to Use Tools for Creating Multimodal AgentsLLaVA-Plus trains a large multimodal model with visual instruction tuning to plan tool use. Each turn gives thoughts, a JSON action list and a reply; tool outputs go back into the chat. Trained on tool-use dialogues plus LLaVA data. Author-reported: LLaVA-Bench In-the-Wild 69.5 vs 57.1 for LLaVA; MM-Vet 13B 35.0 vs 32.5.Paper · 2023-11-07Do LLMs exhibit human-like response biases? A case study in survey designTests whether LLMs show the response biases people show in survey wording: acquiescence, allow/forbid, response order, odd/even scales and opinion floating. Each bias is a pair of questions, and the shift in answer distribution is compared with shifts from non-bias typos. Nine models; most fail to match humans, and RLHF chat models react more to typos (author-reported).Paper · 2023-11-06GLaMM: Pixel Grounding Large Multimodal ModelGLaMM writes a text answer and attaches a segmentation mask to each object phrase it names. It uses a CLIP image encoder, a 7B LLM with LoRA, a region encoder for user boxes and a SAM-based mask decoder, trained on the new GranD dataset (7.5M concepts). Introduces the GCG task. Author-reported gains on GCG and refCOCO-family segmentation over LISA.Paper · 2023-11-06Inner-IoU: More Effective Intersection over Union Loss with Auxiliary Bounding BoxInner-IoU computes the IoU term of a box regression loss on auxiliary boxes, which are the predicted and ground-truth boxes scaled by a ratio about their centres. Ratios below 1 suit high-IoU samples, above 1 low-IoU ones, and no new loss term is added. Author-reported gains on VOC (YOLOv7-tiny) reach +1.21 mAP50:95; on AI-TOD (YOLOv5s) +1.07 AP50.Paper · 2023-11-04Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal LocomotionIFM clones a DDP-based MPC expert into a small MLP with DAgger, then finetunes it with PPO on rough, step and cliff terrains under tight exploration limits. Simulation: better survival than MPC on rough, slippery and conveyor terrains. Mini Cheetah: IFM(CR) climbs a 7.5 cm step in 9 of 10 trials. Compute 0.0154 s to 0.0010 s (author-reported).Paper · 2023-10-31ChipNeMo: Domain-Adapted LLMs for Chip DesignAdapts LLaMA2 to chip design with a domain-extended tokenizer, continued pretraining on internal and public chip data, SFT or SteerLM alignment, and a fine-tuned e5 retriever for RAG. Author-reported: ChipNeMo-70B beats GPT-4 on the engineering chatbot and EDA-script benchmarks and is competitive on bug summaries.Paper · 2023-10-31Unlearn What You Want to Forget: Efficient Unlearning for LLMsTrains small unlearning layers on a frozen T5 model with a selective teacher-student KL objective, then merges them in closed form to handle a sequence of deletion requests. Author-reported on IMDB (T5-base, 0.5% forgotten): forget-set accuracy 65.7 vs 92.5 for re-training, at 1200 s vs 6685 s. The official repo has no unlearning code at the pinned commit.Paper · 2023-10-26ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI ConversationToxicChat is a 10,166-prompt toxicity benchmark built from real first-turn user queries to the Vicuna demo, labelled by a Perspective-assisted human pipeline with an extra jailbreaking label. Classifiers trained on other toxicity datasets transfer poorly to it. A RoBERTa-base trained on ToxicChat reaches F1 72.1 on its test half (author-reported).Paper · 2023-10-23SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical ImagesPromptable segmentation for 3D medical volumes: a fully 3D ViT encoder, a point-prompt encoder and a 3D mask decoder, trained from scratch on SA-Med3D-140K (22K volumes, 143K masks) in two stages. Author-reported Dice: 76.27% overall with one 3D point, against 36.83% for SAM-Med2D with N points (one per slice). With 10 points, 80.71%.Paper · 2023-10-23Unleashing the potential of prompt engineering for large language modelsSurvey of prompt engineering for LLMs and vision-language models. Covers basic prompting, chain-of-thought and other reasoning methods, automatic prompt optimization, CoOp, CoCoOp and MaPLe, evaluation, prompt-based security risks and future directions. It presents no new method or experiments; the numbers it mentions come from cited work.Paper · 2023-10-20Towards Understanding Sycophancy in Language ModelsMeasures sycophancy in five assistants across feedback, answer, flip-flop and poem-attribution tasks, then tests whether human and preference-model (PM) judgments favour it. Author-reported: feedback turns more positive when the user says they like the text; Claude 1.3 admits a mistake on 98% of questions; the Claude 2 PM prefers sycophantic over baseline truthful replies 95% of the time.Paper · 2023-10-18Sociotechnical Safety Evaluation of Generative AI SystemsA three-layer framework for safety evaluation of generative AI: capability (the model), human interaction (real users) and systemic impact (society, economy, environment). The authors map 2023 evaluations to six harm areas, modalities and layers, and find gaps: few evaluations of non-text output, human interaction or systemic impact. No experiments of their own; survey data is author-reported.Paper · 2023-10-18To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For NowUnlearnDiffAtk attacks text-to-image diffusion models that were unlearned to refuse a concept. It prepends 3 to 5 optimised tokens to the prompt, scoring them with the victim's own denoising error on a target image, so no auxiliary model is needed. Author-reported: nudity bypass of ESD at 76.05% vs 69.71% for P4D, in 26.29 vs 34.70 min per prompt.Paper · 2023-10-17EvalCrafter: Benchmarking and Evaluating Large Video Generation ModelsEvalCrafter is a 700-prompt text-to-video benchmark scored by 17 automatic metrics in four aspects: visual quality, text-video alignment, motion and temporal consistency. Per-aspect linear models fitted to 1-5 user ratings give a final score. Per the authors, the fitted scores beat a plain metric average on held-out Spearman correlation in all four aspects.Paper · 2023-10-17GenEval: An Object-Focused Framework for Evaluating Text-to-Image AlignmentGenEval checks a text-to-image output against each prompt's metadata: an object detector (Mask2Former, 80 COCO classes) finds objects, a CLIP ViT-L/14 classifier checks colours, and counts and relative positions are tested per image. The six task scores are averaged. Author-reported: IF-XL scores 0.61; position (best 0.15) and attribute binding (best 0.35) remain hard.Paper · 2023-10-17Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingMeasures how far LLM accuracy moves when only the formatting of a few-shot prompt changes, across grammar-generated formats that keep the meaning. FormatSpread uses Thompson sampling to estimate the best and worst format within a budget, without model weights. Headline: spreads up to 76 accuracy points (LLaMA-2-13B), median 7.5 points across tasks.Paper · 2023-10-17Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionSelf-RAG trains one LM to emit reflection tokens: whether to retrieve, whether a passage is relevant, whether its output is supported by the passage, and how useful the answer is. At inference, segment-level beam search weighs these tokens. Llama 2 7B and 13B versions beat ChatGPT and Llama2-chat on several tasks and gain citation precision on ASQA (author-reported).Paper · 2023-10-16Large Models for Time Series and Spatio-Temporal Data: A Survey and OutlookSurvey of large language models and pre-trained foundation models for time series and spatio-temporal data. Proposes a taxonomy: time series (LM4TS) and spatio-temporal (LM4STD) branches, each split into LLM and PFM approaches, then general and domain-specific. Catalogues methods, datasets and tools. No experiments; coverage claims are the authors' own.Paper · 2023-10-14Large Language Model UnlearningUnlearning for LLMs: gradient ascent on harmful, copyrighted or hallucinated answers, plus a random-answer loss and a KL term on normal data to keep utility. On OPT-1.3B, OPT-2.7B and Llama 2 7B the harmful rate falls to 1-6% from 47-54% (author-reported), and it uses about 2% of the RLHF compute.Paper · 2023-10-13"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference LettersMeasures gender bias in recommendation letters from ChatGPT and Alpaca, given a name and occupation (CLG) or a biography (CBG). Word choice is scored with odds ratios and WEAT, tone with formality, positivity and agency classifiers plus t-tests, and hallucinated sentences with NLI. Male-favouring tone gaps are significant; hallucinated text carries them further.Paper · 2023-10-13Open X-Embodiment: Robotic Learning Datasets and RT-X ModelsOpen X-Embodiment pools 60 robot datasets from 22 embodiments into one RLDS format and trains RT-1-X and RT-2-X policies on a shared action space. Author-reported: RT-1-X beats each dataset's own baseline on 4 of 5 small-data domains, and RT-2-X raises emergent-skill success from 27.3% to 75.8%.Paper · 2023-10-12Jailbreaking Black Box Large Language Models in Twenty QueriesPAIR is a black-box jailbreak method: an attacker LLM proposes a prompt, reads the target's reply and a judge score, and refines the prompt over a few rounds, run as parallel streams. Author-reported on JailbreakBench: 88% jailbreak rate on Vicuna-13B, 73% on Gemini-Pro, 48-51% on GPT-3.5/4, with 10 to 56 queries per success.Paper · 2023-10-11NoMaD: Goal Masked Diffusion Policies for Navigation and ExplorationNoMaD trains one Transformer diffusion policy that either sees or masks a goal image, so one model explores unseen places and reaches a goal once it is found. A denoising U-Net samples 2D action sequences. Author-reported on a real robot: 98% exploration success with 19M parameters, against 77% for the best published baseline with 335M.Paper · 2023-10-11State of the Art on Diffusion Models for Visual ComputingState-of-the-art report (a survey, no experiments of its own) on diffusion models for images, video, 3D and 4D. It gives the SDE/ODE view of diffusion, Euler–Maruyama sampling, the latent-diffusion design of Stable Diffusion, conditioning and guidance, editing and inversion, then datasets, metrics and open problems. Its numbers are dataset sizes and training costs quoted from cited works.Paper · 2023-10-10Mistral 7BMistral 7B is a 7-billion-parameter Transformer language model that uses grouped-query attention, sliding-window attention (window 4096) and a rolling-buffer KV cache for cheaper inference, plus an instruction-tuned variant. Author-reported: it beats Llama 2 13B on all but one benchmark row and Llama 1 34B on reasoning, math and code.Paper · 2023-10-10TopoMLP: A Simple yet Strong Pipeline for Driving Topology ReasoningDriving topology: a PETR-style 3D lane detector (Bezier control points, 300 queries) and a Deformable-DETR traffic detector, optionally with YOLOv8 proposals, feed small MLP heads that predict lane-lane and lane-traffic links. Author-reported: 38.2 OLS with ResNet-50 on OpenLane-V2 subset A, 41.2 with YOLOv8 proposals.Paper · 2023-10-09Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity AnalysisProposes BasicTS+, a benchmark with one training pipeline and re-normalized metrics for multivariate time series forecasting, plus a split of datasets by temporal patterns and spatial indistinguishability. Benchmarks over 45 models; spatial modelling helps only on indistinguishable data.Paper · 2023-10-05Improved Baselines with Visual Instruction TuningLLaVA-1.5 keeps LLaVA's simple design (frozen CLIP ViT, connector, full LLM finetune) but swaps the linear connector for a 2-layer MLP, uses CLIP-ViT-L at 336px, and adds academic VQA data with explicit response-format prompts. With 558K pretrain + 665K public instruction samples, the 13B model is trained in about a day on 8 A100s and reaches state-of-the-art on 11 benchmarks (author-reported).Paper · 2023-10-04From Words to Watts: Benchmarking the Energy Costs of Large Language Model InferenceBenchmarks LLaMA 7B, 13B and 65B inference on V100 and A100 GPUs, sharded across up to 32 GPUs, with Alpaca and GSM8K prompts. Reports tokens/s and GPU energy per second, per token and per response. Author-reported: A100 is about 2x V100 for 7B; capping the 65B run's A100s at 175 W cut energy about 23% for about 7% more time (GSM8K only).Paper · 2023-10-03Time-LLM: Time Series Forecasting by Reprogramming Large Language ModelsKeeps a Llama-7B backbone frozen and trains only small input and output layers. Series patches are cross-attended onto learned text prototypes, a prompt with dataset context and statistics is prepended, and the patch outputs are projected to the horizon. Author-reported: best MSE on 7 of 8 long-term benchmarks, about 12% average gain over GPT4TS, strong few-shot and zero-shot results.Paper · 2023-09-28Qwen Technical ReportQwen is a family of autoregressive Transformer LLMs (1.8B, 7B, 14B) pretrained on up to 3T tokens. Chat models are aligned by SFT and PPO-based RLHF; code and math variants are added. The 14B base reports 66.3 MMLU (5-shot) and 72.1 C-Eval (5-shot), author-reported.Paper · 2023-09-28Text-to-3D using Gaussian SplattingGSGEN generates 3D assets from text with 3D Gaussians. It starts from a Point-E point cloud, then runs a geometry stage (2D SDS plus a point-cloud SDS on Gaussian centres) and an appearance stage (2D SDS, penalties, compactness-based densification). Results are qualitative: fewer Janus artefacts and finer detail than baselines, about 40 minutes per asset on one NVIDIA 3090.Paper · 2023-09-21A Machine Learning-oriented Survey on Tiny Machine LearningPRISMA systematic review of TinyML (machine learning on microcontroller-class devices), January 2018 to September 2023, with 172 eligible papers. Sorts the work into three workflows, a taxonomy of model optimization, model design and learning algorithms, and hardware and software tools. No experiments; the results are review counts and popularity labels.Paper · 2023-09-19Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher EducationSystem paper for VirtualTA (AIIA), a Canvas-integrated course chatbot. Course files and Whisper lecture transcripts are cut into ~800-character chunks and embedded with text-embedding-ada-002; queries are classified, matched by cosine similarity (top 10, kept above 0.75) and answered by GPT-3.5 models. Adds flashcards, quizzes, auto-grading and homework detection. No quantitative evaluation.Paper · 2023-09-18SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language ModelsPrompts an LLM with Python-style few-shot examples through four stages (decomposition, coalition formation, allocation, code) to turn one instruction into code for a multi-robot team in AI2-THOR. Author-reported GPT-4 success rate: 1.00 on elemental tasks, 0.62 to 0.71 on the other three categories of a 36-task benchmark.Paper · 2023-09-15Sparse Autoencoders Find Highly Interpretable Features in Language ModelsTrains a sparse autoencoder (ReLU encoder, L1 penalty) on Pythia residual-stream activations to learn an overcomplete dictionary of features. Author-reported: the features score higher on automated interpretability than neurons, PCA, ICA and random directions, and few of them are needed to localise the indirect-object-identification behaviour.Paper · 2023-09-14Adapted Large Language Models Can Outperform Medical Experts in Clinical Text SummarizationAdapts eight LLMs to four clinical summarization tasks (six datasets) by in-context examples or QLoRA, scores them with NLP metrics, and runs a ten-physician reader study. The best setup, GPT-4 with the most in-context examples, gives summaries rated equivalent (45%) or better (36%) than expert summaries (author-reported).Paper · 2023-09-12A Survey of Hallucination in "Large" Foundation ModelsSurvey of hallucination (generated content that strays from fact or is fabricated) in large foundation models across text, image, video and audio. Sorts 28 works by whether they detect or mitigate hallucination, plus their tasks, datasets and metrics, and lists open directions. It runs no experiments and reports no new results.Paper · 2023-09-12Efficient Memory Management for Large Language Model Serving with PagedAttentionPagedAttention stores each sequence's KV cache in fixed-size blocks (default 16 tokens) that need not be contiguous, mapped through per-sequence block tables with reference counts and copy-on-write, like OS virtual memory. vLLM, built on it, reports 2-4x higher serving throughput than FasterTransformer and Orca at the same latency.Paper · 2023-09-07Large Language Models as OptimizersOPRO uses an LLM as a black-box optimizer: each step shows it past solutions with their scores and asks for better ones, which are scored and added back. It finds optima on some small linear-regression and TSP instances. For prompt optimization, the best instructions beat human-written prompts by up to 8% on GSM8K and up to 50% on BBH (author-reported).Paper · 2023-09-05A Survey of Imitation Learning: Algorithms, Recent Developments, and ChallengesSurvey of imitation learning (IL) for robots and AI. It covers behavioural cloning and its covariate-shift fixes, inverse RL, adversarial IL, imitation from observation, and open challenges: imperfect demonstrations and domain mismatch. One table lists 66 works by learning type and online-expert use. It reports no experiments of its own.Paper · 2023-09-02Explainability for Large Language Models: A SurveySurvey of explainability for Transformer LLMs, split by training paradigm. Fine-tuned models: local methods (feature attribution, attention, examples, natural-language explanations) and global methods (probing, neurons, concepts, circuits). Prompted models: in-context learning, chain-of-thought, representation engineering. Also covers evaluation metrics and open challenges. No new experiments.Paper · 2023-08-31The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language VariantsBelebele is a multiple-choice reading-comprehension benchmark: 900 questions on 488 FLORES-200 passages, written in English and translated by experts into 122 language variants. Human accuracy is 97.6%. Author-reported: LLaMA 2 70B scores 90.9% on English in 5-shot, but averages 48.0 over all languages. The best multilingual fine-tuned model averages 60.2.Paper · 2023-08-24Code Llama: Open Foundation Models for CodeCode Llama continues Llama 2 training on code, then adds infilling, long-context fine-tuning (RoPE base raised to 1e6, 16k sequences) and instruction tuning on self-generated, unit-tested data. Sizes 7B to 70B, three variants. Author-reported HumanEval pass@1 reaches 67.8 (Instruct 70B) and MBPP 65.6 (Python 70B), greedy decoding.Paper · 2023-08-24Dense Text-to-Image Generation with Attention ModulationDenseDiffusion adapts a frozen Stable Diffusion model to dense captions and layouts without training. In the first 30% of denoising steps it raises attention scores between words and pixels of the same region and lowers the others, scaled by score range and region size. Author-reported on 250 MS-COCO captions: SOA-I 77.61 vs 73.08 for Stable Diffusion; layout IoU 34.99 vs 23.76 for SD-Pww.Paper · 2023-08-22A Survey on Large Language Model based Autonomous AgentsSurvey of LLM-based autonomous agents. Proposes a unified framework of four modules (profile, memory, planning, action) and a taxonomy of how agent capabilities are acquired (fine-tuning, prompt engineering, mechanism engineering). Covers applications in social science, natural science and engineering, and subjective and objective evaluation. It is a survey: it reports no experiments of its own.Paper · 2023-08-21Can Language Models Learn to Listen?Predicts a listener's facial motion from the speaker's timed transcript alone. A VQ-VAE turns motion into 256 tokens, which GPT-2 Medium (text-pretrained) generates interleaved with words. Author-reported: best FD, P-FD and L2 Affect on the Trevor Noah split; 70.1% human preference over retrieval.Paper · 2023-08-21On the Adversarial Robustness of Multi-Modal Foundation ModelsImperceptible l-infinity perturbations (radius 1/255 or 4/255) on images make OpenFlamingo (9B) output a chosen caption or off-topic text. White-box APGD attacks the query image and optionally the few-shot context images. At 4/255 the targeted attack reaches 100% success for one target with 500 steps; untargeted COCO CIDEr drops to 1.69 (0-shot, all images attacked).Paper · 2023-08-20ExpeL: LLM Agents Are Experiential LearnersExpeL lets a frozen LLM agent improve across tasks without weight updates. It collects success and failure trajectories on training tasks with Reflexion-style retries, has the LLM distil them into written insights, and at test time adds those insights and similar past successes to the prompt. Author-reported, gpt-3.5-turbo policy: HotpotQA 39% vs 28% for ReAct, ALFWorld 59% vs 40%.Paper · 2023-08-17An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningMeasures catastrophic forgetting when instruction-tuning LLMs (BLOOMZ 1.1b-7.1b, mT0, LLaMA-7b, Alpaca-7b) on five tasks in sequence. Forgetting is scored as the average relative drop in knowledge, reasoning, reading and bias benchmarks. Author-reported: forgetting appears in every model tested, BLOOMZ forgets less than mT0, and Alpaca forgets less than LLaMA.Paper · 2023-08-17EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language UnderstandingEgoSchema is a multiple-choice video QA benchmark of about 5,000 questions on three-minute Ego4D clips. Questions are generated by LLMs, filtered by rules and a blind-answer test, then curated by humans. The temporal certificate median is about 100 seconds. Zero-shot video-language models score 19.6-32.1% against 20% chance, while humans reach about 76% (author-reported).Paper · 2023-08-16AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent ConversationAutoGen is a framework where agents backed by LLMs, humans or tools exchange messages to solve a task. Developers set up agents and program the conversation with auto-reply functions, natural language or code. Author-reported: 69.48% on the full MATH test set against 55.18% for vanilla GPT-4, and 69% average success on ALFWorld for a three-agent design against 54% for ReAct.Paper · 2023-08-13Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in MedicineFew-shot prompts that ask GPT-3.5 and GPT-4 to reason like clinicians (intuitive, analytic, differential and Bayesian styles) on free-response USMLE-based diagnosis questions. GPT-4 scores 72-78% against 76% for chain-of-thought, with no significant difference; GPT-3.5 loses 4-8 points on three of the four styles (author-reported).Paper · 2023-08-11Diverse Data Augmentation with Diffusions for Effective Test-time Prompt TuningDiffTPT tunes four CLIP context tokens per unlabeled test image. It adds 63 Stable Diffusion image variations to the usual crop and flip views, keeps the 80% most cosine-similar views and then the 30% lowest-entropy ones. Author-reported zero-shot gains over TPT on ImageNet variants and 10 fine-grained datasets are 0.7 to 2.9 points on the averages printed in the tables.Paper · 2023-08-06Spanish Pre-trained BERT Model and Evaluation DataTrains BETO, a BERT-Base-sized encoder (110M parameters) on about 3 billion words of Spanish text, in cased and uncased versions with whole-word and dynamic masking, and compiles Spanish tasks as GLUES. Author-reported: beats multilingual BERT on most tasks and sets new state of the art on POS and MLDoc, but trails it on some QA settings.Paper · 2023-08-01MetaGPT: Meta Programming for A Multi-Agent Collaborative FrameworkMetaGPT gives LLM agents the roles of a software company (product manager, architect, project manager, engineer, QA engineer). They pass structured documents through a shared message pool with role subscriptions, and the engineer runs and debugs its code. Author-reported: 85.9% HumanEval and 87.7% MBPP Pass@1, and higher executability than ChatDev on seven SoftwareDev tasks.Paper · 2023-07-28Med-HALT: Medical Domain Hallucination Test for Large Language ModelsMed-HALT is a benchmark for hallucinations in medical LLMs. Reasoning tests (false confidence, none-of-the-above, fake questions) use multinational exam items; four PubMed recall tests check titles, PMIDs and links. Answers score +1 if correct and -0.25 if wrong. Author-reported: Llama-2 70B leads false-confidence accuracy (42.21%); Falcon 40B scores 99.89% on fake questions.Paper · 2023-07-27Open Problems and Fundamental Limitations of Reinforcement Learning from Human FeedbackSurvey and position paper on RLHF. It splits open problems into challenges with human feedback, the reward model, the policy and joint training, and labels each as tractable or fundamental. It then reviews complementary safety methods and proposes disclosure items for auditing. It runs no experiments; empirical figures are cited from other work.Paper · 2023-07-26Towards Generalist Biomedical AIMed-PaLM M fine-tunes PaLM-E (12B, 84B, 562B) on MultiMedBench, 14 biomedical tasks spanning text, radiology, pathology, dermatology, mammography, chest X-ray and genomics, with one set of weights and text instructions. Author-reported: competitive with or above prior SOTA on many tasks; radiologists preferred its chest X-ray reports to references in up to 40.50% of cases.Paper · 2023-07-25Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future OpportunitiesSurvey and benchmark of decision-focused learning, which trains a predictor on the regret of the decision a solver returns, not on prediction error. Covers gradient-based methods (differentiated optimality conditions, smoothed LPs, perturbations, surrogate losses) and gradient-free ones. Benchmarks 11 methods on 7 problems; SPO is described as most robust. Author-reported.Paper · 2023-07-24Evaluating the Ripple Effects of Knowledge Editing in Language ModelsDefines six "ripple effect" criteria for knowledge editing: facts that must change or stay after an edit. Builds RippleEdits from Wikidata and tests ROME, MEMIT, MEND and an in-context prompt on GPT-2 XL, GPT-J, GPT-NeoX and LLaMA-7B. Author-reported: weight editors often miss implied facts; in-context editing scores best on average.Paper · 2023-07-19Challenges and Applications of Large Language ModelsAn opinionated survey, not a method. It lists open LLM challenges in three groups (design, behavior, science), from data duplication and tokenizer limits to hallucination, brittle evaluation and irreproducible APIs, and surveys LLM applications in chatbots, biology, programming, law, medicine, reasoning, robotics and social science. It runs no experiments of its own.Paper · 2023-07-18RepViT: Revisiting Mobile CNN From ViT PerspectiveRepViT modernises MobileNetV3-L with design choices from lightweight ViTs: split token and channel mixers, a 2x-expansion FFN, a convolutional stem, deeper downsampling, cross-block SE, and re-parameterised depthwise convs. Author-reported: 80.0% ImageNet top-1 at 1.0 ms on iPhone 12 (RepViT-M1.0, distilled) and 83.7% at 2.3 ms (RepViT-M2.3).Paper · 2023-07-13Self-regulating Prompts: Foundational Model Adaptation without ForgettingPromptSRC trains deep vision-language prompts for CLIP with the usual cross-entropy loss plus three self-regularisers: L1 and KL agreement with the frozen CLIP features and logits, Gaussian-weighted averaging of prompts over training epochs, and text-template ensembling on the frozen side. Author-reported base-to-novel harmonic mean over 11 datasets: 79.97 vs 78.55 for MaPLe.Paper · 2023-07-12A Comprehensive Overview of Large Language ModelsA survey of large language models (LLMs) with 10B+ parameters. It covers the transformer building blocks, pre-trained and fine-tuned models, efficiency methods, datasets, evaluation, applications and open challenges. It tabulates architecture and optimizer settings for many LLMs. It runs no experiments of its own, so all numbers are copied from cited papers.Paper · 2023-07-11Objaverse-XL: A Universe of 10M+ 3D ObjectsObjaverse-XL is a web-crawled set of about 10.2M deduplicated 3D objects from five sources, each rendered from 12 views and filtered. Zero123 trained on it (Zero123-XL) generalizes better zero-shot; PixelNeRF pretrained on it gains PSNR on DTU and ShapeNet. Author-reported: Zero123-XL PSNR rises from 18.2 to 19.9 on Google Scanned Objects after alignment finetuning.Paper · 2023-07-10AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation SystemVision-based teleoperation for dexterous arm-hand robots. MediaPipe hand detection, optional depth, multi-camera fusion, optimization-based retargeting to robot hand joints, and CuRobo arm motion. Author-reported: higher real-robot success than Telekinesis on 8 of 10 tasks; better imitation learning than a simulator-specific baseline on 5 of 6 SAPIEN settings.Paper · 2023-07-10Large Language Models as General Pattern MachinesPrompts a frozen LLM with few-shot sequences, including ones written in random tokens, and reports it completes ARC grids (85 of 800 for text-davinci-003), PCFG transformations and sinusoids. It then uses reward-labelled trajectories in context for simple robot trajectory and CartPole control. All results are author-reported.Paper · 2023-07-10RoCo: Dialectic Multi-Robot Collaboration with Large Language ModelsEach robot gets its own LLM agent. The agents discuss in a dialog and propose one sub-task per robot, and the simulator checks the plan (task rules, reach, inverse kinematics, collisions, waypoint spacing), returning failures as feedback for re-planning. A multi-arm RRT planner then produces joint trajectories. Author-reported with GPT-4 on six MuJoCo tasks: 0.44 to 0.95 success per task.Paper · 2023-07-07A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly DetectionSurvey of graph neural networks for time series (GNN4TS) across forecasting, classification, anomaly detection and imputation. It gives a task taxonomy and a spatial, temporal and architecture taxonomy of spatial-temporal GNNs, with each task's objective, graph-construction rules and application areas. It runs no experiments, so it reports no new numbers.Paper · 2023-07-06A Survey on Evaluation of Large Language ModelsSurvey (no new experiments) of how large language models are evaluated, organised as what to evaluate (tasks), where (about 46 benchmarks and datasets) and how (automatic metrics and human judging). It summarises where LLMs succeed and fail, and lists open challenges such as dynamic benchmarks and trustworthy evaluation. Author-reported catalogue; there is no method to implement.Paper · 2023-07-05Loss Functions and Metrics in Deep LearningSurvey of loss functions and performance metrics for regression, classification, computer vision, NLP and retrieval-augmented generation. For each loss or metric it gives the formula, its properties and where it is used, and it recommends weighted sums for multi-loss training. It has no experiments and no benchmark numbers of its own.Paper · 2023-07-04Robots That Ask For Help: Uncertainty Alignment for Large Language Model PlannersKnowNo turns an LLM planner's next-step choices into multiple-choice options, scores them with its own token likelihoods and applies split conformal prediction on about 400 calibration scenarios. The robot acts when the prediction set has one option and asks a human otherwise, which gives a user-chosen task-success level. Hardware help rate: 0.67 vs 0.81 for a baseline.Paper · 2023-07-04SDXL: Improving Latent Diffusion Models for High-Resolution Image SynthesisSDXL is a latent text-to-image diffusion model with a UNet about three times larger than Stable Diffusion 1.x/2.x, two text encoders, and size, crop and aspect-ratio conditioning instead of discarding or cropping training images. An optional SDEdit-style refiner adds detail. Author-reported user study: refiner-enhanced SDXL wins 48.44% of votes against 7.91% for SD 1.5 and 6.71% for SD 2.1.Paper · 2023-07-01Residual-based attention in physics-informed neural networksResidual-based attention (RBA) gives each PINN collocation point a weight that is a decaying sum of its normalised PDE residual, with no gradients or extra networks. Author-reported: 5.8e-5 relative L2 on 1D Allen-Cahn (five-seed mean), 8.21e-6 on 2D Helmholtz with Fourier features, and about one order of magnitude faster early convergence than vanilla PINNs.Paper · 2023-06-29One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape OptimizationTurns one photo into a 360-degree textured mesh in one feed-forward pass. Zero123 generates 32 posed views (8 anchors, 4 nearby views each), a LoFTR-based search estimates the input elevation, and an SDF cost-volume network built on SparseNeuS fuses the views. Author-reported: F-score 83.5 averaged over GSO and Objaverse (best), CLIP similarity 78.1 (second to Shap-E), about 45 s on an A100.Paper · 2023-06-27Shikra: Unleashing Multimodal LLM's Referential Dialogue MagicShikra is a multimodal LLM that takes image regions as input and writes image coordinates as plain numbers in its text output. It uses a frozen CLIP ViT-L/14, one linear projection and Vicuna-7B or 13B, with no extra vocabulary, position encoder or detector. Author-reported: 87.01 on RefCOCO val (Shikra-7B) and the best PointQA scores in the paper.Paper · 2023-06-26MotionGPT: Human Motion as a Foreign LanguageMotionGPT turns 3D human motion into discrete codes with a VQ-VAE (512 codes, 4x temporal downsampling) and adds them to the vocabulary of Flan-T5. It pretrains on text and motion, then instruction-tunes on prompts for text-to-motion, motion-to-text, prediction and in-between. Author-reported: HumanML3D text-to-motion FID 0.232, but T2M-GPT reports 0.116.Paper · 2023-06-23A Survey on Multimodal Large Language ModelsSurvey of multimodal large language models (MLLMs). Splits an MLLM into a modality encoder, a pre-trained LLM and a connector, then covers three training stages (pre-training, instruction tuning, alignment tuning), evaluation, hallucination, multimodal in-context learning, multimodal chain-of-thought and LLM-aided visual reasoning. It runs no experiments of its own.Paper · 2023-06-20DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT ModelsDecodingTrust is a benchmark that tests GPT-4 and GPT-3.5 on eight trust areas (toxicity, stereotype bias, adversarial and OOD robustness, robustness to adversarial demonstrations, privacy, machine ethics, fairness) with benign and adversarial prompts. Headline: GPT-4 is usually more trustworthy on standard benchmarks, yet it follows jailbreak system prompts more readily than GPT-3.5.Paper · 2023-06-20Textbooks Are All You Needphi-1 is a 1.3B-parameter Transformer for Python code. It is pretrained on about 7B tokens of filtered web code and GPT-3.5-written textbooks, then fine-tuned on about 180M tokens of generated exercises. Author-reported: 50.6% pass@1 on HumanEval and 55.5% on MBPP.Paper · 2023-06-19RemoteCLIP: A Vision Language Foundation Model for Remote SensingContinues CLIP pretraining on a remote sensing image-text set grown about 12x by turning detection boxes and segmentation masks into five captions per image (B2C, M2B). Author-reported: ViT-L-14 reaches 50.52 and 36.35 mean recall on RSITMD and RSICD retrieval, and ViT-B-32 gains +6.39 zero-shot average accuracy over CLIP on 12 datasets.Paper · 2023-06-14NodeFormer: A Scalable Graph Structure Learning Transformer for Node ClassificationNodeFormer is a graph Transformer for node classification in which every node passes messages to every other node in each layer. A kernelized Gumbel-Softmax with random features keeps this linear in node count and learns a sparse latent graph per layer; input edges enter as a bias and a likelihood term. Author-reported: 77.45 ROC-AUC on OGB-Proteins, 87.85% on Amazon2M.Paper · 2023-06-11Digital Twin-Enhanced Wireless Indoor Navigation: Achieving Efficient Environment Sensing with Zero-Shot Reinforcement LearningTrains a PPO navigation policy for mmWave indoor navigation with physics terms added to the RL reward: link-state monotonicity, following the AoA of the strongest path, and an SNR-gradient term. Author-reported: on six unseen Gibson maps the normalised path length in 2+-NLOS is 2.5-2.8 versus 4.5-5.1 for end-to-end RL, and success is 1.0 versus 0.40-0.45 on two maps.Paper · 2023-06-08Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language ModelsVideo-ChatGPT feeds frames through a frozen CLIP ViT-L/14 encoder, pools them into temporal and spatial tokens, and maps them into Vicuna-7B with one trainable linear layer. That layer is tuned on 100K video-instruction pairs from human and GPT-assisted annotation. Author-reported: 2.40 correctness on a 1-5 GPT-judged benchmark, the top score among the compared video chat models.Paper · 2023-06-07PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial PromptsPromptRobust attacks the instruction text, not the input sample. Seven text attacks (typos, word swaps, appended distractors, translation round trips) are run on four prompt types across 13 datasets and six LLMs, scored by Performance Drop Rate. Word-level swaps hurt most (author-reported average drop 0.31-0.33); GPT-4 and Flan-UL2 are the most robust.Paper · 2023-06-06Emergent Correspondence from Image DiffusionDIFT: take a pretrained image diffusion model, noise a real image to level t, run one denoiser pass and keep an intermediate U-Net layer, averaged over noise draws. Nearest-neighbour matching on these features finds correspondences with no training. Author-reported: Stable Diffusion features reach 52.9 PCK per image on SPair-71k, against 33.9 for DINO and 38.4 for OpenCLIP.Paper · 2023-06-06Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelInference-Time Intervention (ITI) finds attention heads whose linear probes separate true from false answers, then shifts those heads' outputs along a truthful direction at every decoding step, with no retraining. On TruthfulQA, Alpaca's true*informative score rises from 32.5% to 65.1%, author-reported; a strength knob trades truthfulness for informativeness.Paper · 2023-06-05LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot LearningLIBERO is a simulated robot-manipulation benchmark for lifelong learning: 130 language-conditioned tasks in four suites, with 50 teleoperated demonstrations each. It compares three policy networks and five algorithms, including ER, EWC and PackNet. Author-reported: sequential finetuning has the best forward transfer, and naive supervised pretraining hurts downstream learning.Paper · 2023-06-05LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative FusionTwo-stage LLM ensemble. PairRanker encodes an input with two candidate outputs, learns pairwise preferences, and ranks all candidates from pairwise scores. GenFuser (Flan-T5-XL) fuses the top 3 into one answer. On MixInstruct (11 open LLMs) the fused output reaches GPT-Rank 3.01 vs 3.90 for the best single model (author-reported).Paper · 2023-06-05Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM AgentsProposes a conceptual framework for LLM multi-agent systems. Agents are tuples of model, role, state, creation and halting rights; plugins are tools; an oracle and a supervisor agent give feedback and halt loops. No experiments: it discusses Auto-GPT, BabyAGI, Gorilla and two design scenarios, and reports no quantitative result.Paper · 2023-06-04Auto-GPT for Online Decision Making: Benchmarks and Additional OpinionsPuts chat LLMs (GPT-4, GPT-3.5, Claude, Vicuna) in an Auto-GPT-style loop with tool calls on WebShop and ALFWorld. Adds Additional Opinions: a trained imitation model's suggested action goes into the prompt, with no fine-tuning. Author-reported: GPT-4 with five opinions reaches 0.320 success on 50 WebShop tasks, vs 0.227 for the imitation baseline.Paper · 2023-06-01A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnosticsIRENE turns a chest radiograph, chief-complaint words, lab values and demographics into one token sequence. Two bidirectional image-text attention blocks, then ten self-attention blocks, feed a classifier. Author-reported: mean AUROC 0.924 on eight pulmonary diseases (Perceiver 0.858); mean AUPRC 0.592 on three COVID-19 outcomes (early fusion 0.521).Paper · 2023-05-29Direct Preference Optimization: Your Language Model is Secretly a Reward ModelDirect Preference Optimization (DPO) fine-tunes a language model on pairwise human preferences with one logistic loss on policy-versus-reference log-probability ratios. It needs no separate reward model and no RL sampling loop. Reported to match or beat PPO-based RLHF on sentiment control, TL;DR summarization (GPT-4 judged) and Anthropic-HH dialogue, with models up to 6B parameters.Paper · 2023-05-29Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language ModelsMarked Personas asks an LLM for portraits of intersectional groups and of unmarked defaults (White, man), then finds the words that distinguish each marked group with Fightin' Words log-odds (z > 1.96), with no lexicon. GPT-4 and GPT-3.5 personas contain more racial stereotypes than human-written ones; top words are positive yet show othering (author-reported).Paper · 2023-05-25PandaGPT: One Model To Instruction-Follow Them AllPandaGPT gives Vicuna-13B image, video, audio, depth, thermal and IMU inputs by training only a linear projection from frozen ImageBind embeddings and LoRA on the attention layers, using 160k image-instruction examples. Cross-modal behaviour is shown by examples; the paper reports no benchmark numbers.Paper · 2023-05-25Role-Play with Large Language ModelsPosition paper. It argues that an LLM dialogue agent is best described as role-playing characters, a superposition of simulacra run by a simulator (the base model plus sampling), not as an agent with its own beliefs or goals. It applies this to apparent deception and apparent self-preservation and proposes two behavioural tests. It reports no experiments or measured results.Paper · 2023-05-24BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and EditingBLIP-Diffusion conditions Stable Diffusion v1-5 on a subject image: 16 query outputs from a BLIP-2 multimodal encoder are projected and inserted into the prompt. Two-stage pre-training (alignment, then prompted context generation on synthetic composites) gives zero-shot subject-driven generation and fine-tuning in 40-120 steps. Author-reported DreamBench DINO: 0.594 zero-shot, 0.670 fine-tuned.Paper · 2023-05-24Deep Learning for Survival Analysis: A ReviewSystematic review of 61 deep-learning methods for time-to-event analysis, found by a Web of Science search to 31 Dec 2022 and classified by model class, network architecture, outcome types and feature types. Most methods are single-risk and right-censored; results across papers cannot be compared, so the review ranks nothing.Paper · 2023-05-24Gorilla: Large Language Model Connected with Massive APIsFine-tunes LLaMA-7B to write API calls from natural-language requests, optionally with a retriever whose documentation is in the training prompt so the model follows changed docs. On the authors' APIBench (TorchHub, TensorFlow Hub, HuggingFace), zero-shot Gorilla scores 59.13, 71.68 and 83.79 overall accuracy against GPT-4's 38.70, 19.80 and 18.20 (author-reported).Paper · 2023-05-24LayoutGPT: Compositional Visual Planning and Generation with Large Language ModelsLayoutGPT prompts a text-only LLM, with no training, to write object layouts as CSS-style rules, using retrieved demonstrations and normalized coordinates. A layout-to-image model then renders them. Author-reported: GPT-3.5 reaches 86.33% numerical layout accuracy (human layouts 92.56%); 3D bedroom out-of-bounds 43.26% vs ATISS 49.88%.Paper · 2023-05-24RoMa: Robust Dense Feature MatchingRoMa is a dense feature matcher. A frozen DINOv2 ViT-L/14 gives coarse features, a VGG19 pyramid gives fine ones, and a Gaussian-process encoder with a 5-block Transformer decoder predicts 64x64 anchor probabilities. A robust Charbonnier loss refines the warp. Author-reported: 80.1 WxBS mAA@10px (DKM 58.9), 88.0 IMC2022 mAA@10, 62.6 AUC@5 on MegaDepth-1500.Paper · 2023-05-23Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyMLMulti-objective Bayesian optimization for TinyML networks: a GP surrogate plus small ARS reinforcement-learning policies that propose the next training and pruning settings. It trades accuracy against ROM, RAM and FLOPs under microcontroller limits. Author-reported: higher hypervolume than ParEGO, TurBO, MorBO, NSGA-II and random search in 150 samples on two tasks.Paper · 2023-05-23Grammar-Constrained Decoding for Structured NLP Tasks without FinetuningGrammar-constrained decoding masks an LLM's next-token choices with an incremental parser, so output always follows a formal grammar, with no finetuning. Input-dependent grammars restrict outputs to per-input candidates. Few-shot LLaMA-33B reaches 36.0 F1 on closed IE, 80.3 average accuracy on entity disambiguation, and 54.6 F1 with 100% valid trees on constituency parsing (author-reported).Paper · 2023-05-23Having Beer after Prayer? Measuring Cultural Bias in Large Language ModelsCAMeL: 628 naturally occurring Arabic prompts and 20,368 Arab and Western entities in eight types. The Cultural Bias Score (CBS) measures how often masked and causal LMs prefer Western fillings. Author-reported: on culturally contextualized prompts, average CBS of Arabic and multilingual LMs is 45.5 to 63.5 (0 is ideal). Stories, NER and sentiment show stereotypes and unfairness.Paper · 2023-05-23Improving Factuality and Reasoning in Language Models through Multiagent DebateSeveral copies of one chat model answer a question, then read each other's answers and revise over a few rounds until they tend to agree. Author-reported with gpt-3.5-turbo: debate beats single answers, self-reflection and majority voting on arithmetic, GSM8K and chess, and raises factual accuracy on biographies and MMLU.Paper · 2023-05-23On the Risk of Misinformation Pollution with Large Language ModelsUses GPT-3.5 to write fake passages in four settings and injects them into the retrieval corpora of open-domain QA. Exact match falls 14% to 87% under the three adversarial settings (NQ-1500, CovidNews; author-reported). Three defenses (a fake-text detector, a warning prompt, and voting over reader groups) help only partly.Paper · 2023-05-23QLoRA: Efficient Finetuning of Quantized LLMsQLoRA freezes a pretrained LLM in 4-bit NormalFloat (NF4) with double-quantized block constants, dequantizes to BF16 on the fly, and trains LoRA adapters on every linear layer, with paged optimizers to absorb memory spikes. This lets a 65B model be finetuned on one 48GB GPU, and it matches 16-bit LoRA/full finetuning on GLUE, Super-NaturalInstructions and MMLU (author-reported).Paper · 2023-05-22GQA: Training Generalized Multi-Query Transformer Models from Multi-Head CheckpointsGrouped-query attention (GQA) splits query heads into G groups that each share one key and one value head, sitting between multi-head (G=H) and multi-query (G=1) attention. Multi-head T5 checkpoints are converted by mean-pooling K/V heads, then uptrained for 5% of pre-training steps. GQA-8 T5-XXL averages 47.1 vs 47.2 for MHA at 0.28 s vs 1.51 s per sample.Paper · 2023-05-21Evaluating the Performance of Large Language Models on GAOKAO BenchmarkGAOKAO-Bench: 2,811 questions from China's national college entrance exam (2010-2022) across 10 subjects. Models answer zero-shot at temperature 0.3; objective answers are rule-matched; subjective answers are graded by GPT-4-turbo against teacher rubrics. GPT-4 scores above 400 of 750 converted points; judge-teacher question-level Spearman is 0.825-0.864 (author-reported).Paper · 2023-05-20Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical ReasoningLogic-LM has an LLM translate a logic problem into a formal program (logic programming, first-order logic, CSP or SAT), runs a deterministic solver on it, and can rewrite failing programs using the solver's error message. Author-reported: with GPT-4 it beats standard prompting on all five datasets and chain-of-thought on four.Paper · 2023-05-19LLM-Pruner: On the Structural Pruning of Large Language ModelsLLM-Pruner removes whole dependent groups of LLaMA or Vicuna weights, found by a dependency graph and scored with a gradient-times-weight Taylor estimate on 10 BookCorpus samples, then recovers with LoRA on about 50k Alpaca samples. At 20% of parameters removed, LLaMA-7B keeps 94.97% of its re-run zero-shot average (author-reported).Paper · 2023-05-18Drag Your GAN: Interactive Point-based Manipulation on the Generative Image ManifoldEdits a StyleGAN2 image by dragging handle points to target points. Each step optimises the latent code to move features near the handles, then re-finds each handle by nearest-neighbour search in the same features; no extra network is used. Author-reported: mean landmark distance 2.44 with one point, against 11.64 for UserControllableLT, at 2.0 s per edit.Paper · 2023-05-18LIMA: Less Is More for AlignmentFine-tunes LLaMa 65B with the standard supervised loss on 1,000 curated prompt-response pairs and no RLHF. Human raters judged its answers equal to or better than GPT-4's in 43% of comparisons, and better than DaVinci003 (RLHF-trained) in 65%. The authors argue alignment mainly teaches response style, while knowledge comes from pretraining.Paper · 2023-05-18SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational AbilitiesAdds discrete speech units from a self-supervised speech model to an LLM's vocabulary, so it takes speech in and gives speech out. Three stages: unit pre-training on unlabeled speech, instruction tuning on about 9M speech-text pairs (SpeechInstruct), then LoRA tuning on text-then-speech dialogues. The headline result is qualitative: human-judged examples, no benchmark numbers.Paper · 2023-05-18VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric TasksTreats vision-centric tasks as language: a language-guided tokenizer turns an image into 100 tokens, and a LoRA-tuned Alpaca-7B decodes boxes, points and class names as discrete tokens from a text instruction. One model switches tasks by changing the prompt. Author-reported: 44.6 COCO box AP with ResNet-50 and 60.2 with InternImage-H; 31.0 BLEU-4 on COCO captioning.Paper · 2023-05-17FastComposer: Tuning-Free Multi-Subject Image Generation with Localized AttentionFastComposer personalizes Stable Diffusion v1-5 to several people from one reference photo each, without per-subject fine-tuning. CLIP image embeddings are added to each person's text token, cross-attention is supervised with masks during training, and subject conditioning starts after the layout forms. Author-reported: single-subject identity 51.41% vs 43.37% (Custom Diffusion); 2 s vs 789 s.Paper · 2023-05-17PaLM 2 Technical ReportPaLM 2 is a Transformer language model (sizes and architecture withheld) trained on a more multilingual, deduplicated mixture with a tuned set of pre-training objectives and toxicity control tokens. Scaling runs suggest parameters and tokens should grow at roughly equal rates. Author-reported: BIG-Bench Hard 78.1 vs PaLM 65.2; GSM8K 91.0 vs 74.4, both with self-consistency.Paper · 2023-05-17Preserve Your Own Correlation: A Noise Prior for Video Diffusion ModelsPYoCo finetunes a pretrained text-to-image diffusion model for text-to-video. Instead of i.i.d. noise per frame, it samples correlated noise across frames, either mixed (shared plus per-frame part) or progressive (each frame perturbs the previous one). Author-reported: zero-shot UCF-101 IS 47.76, FVD 355.19; unconditional UCF-101 IS 60.01, FVD 310 at 253M parameters.Paper · 2023-05-17Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsTree of Thoughts (ToT) runs BFS or DFS over intermediate reasoning steps ("thoughts"): the LM proposes candidate next steps, the same LM scores or votes on partial solutions as the search heuristic, and weak branches are pruned or backtracked. No training. With GPT-4, Game of 24 success rises from 4% (CoT) to 74%.Paper · 2023-05-16SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and VerificationSmall draft models propose a token tree per request; the LLM verifies every node in one decoding step using tree attention with a topology-aware causal mask, then keeps the accepted tokens. Author-reported speedups over existing serving systems: 1.5-2.8x distributed, 2.6-3.5x with offloading.Paper · 2023-05-16Towards Expert-Level Medical Question Answering with Large Language ModelsMed-PaLM 2 adapts PaLM 2 to medical QA by instruction tuning on MultiMedQA, and adds ensemble refinement: sample 11 chain-of-thought answers, then write 33 refined answers conditioned on them and take a plurality vote. Author-reported: 86.5% on MedQA (USMLE), a new state of the art at publication. Physicians preferred its long-form answers to physicians' on 8 of 9 pairwise axes. No official code.Paper · 2023-05-15Common Diffusion Noise Schedules and Sample Steps are FlawedCommon noise schedules keep some signal at the last step and samplers skip it, so training and sampling differ. Fixes: rescale to zero terminal SNR, train with v prediction, sample from the last step, rescale CFG. Fine-tuned SD 2.1-base: FID 21.66 vs 23.76 official (10k COCO, DDIM 50 steps). Author-reported.Paper · 2023-05-15From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP ModelsPlaces 14 pretrained LMs on a social and economic political compass by probing them with 62 statements, then pretrains RoBERTa and GPT-2 on left, center and right news and Reddit corpora to see whether their leanings move. Fine-tuned RoBERTa variants differ on hate speech and misinformation detection by target group and news source. Author-reported.Paper · 2023-05-14ULIP-2: Towards Scalable Multimodal Pre-training for 3D UnderstandingULIP-2 pre-trains a 3D point-cloud encoder to align point clouds with rendered views and with language. Each view gets one BLIP-2 caption, chosen by CLIP rank, so no 3D human labels are needed. Image and text encoders are frozen OpenCLIP ViT-G, used for most experiments. Author-reported: 50.6% zero-shot top-1 on Objaverse-LVIS and 84.7% on ModelNet40.Paper · 2023-05-13OCRBench: On the Hidden Mystery of OCR in Large Multimodal ModelsOCRBench is a 1000-question test of OCR skills in large multimodal models: text recognition, scene-text and document VQA, key information extraction and handwritten maths. A reply passes when a ground-truth string appears in it. Author-reported final scores (out of 1000): Gemini 659, GPT4V 645, Monkey 514.Paper · 2023-05-11EfficientViT: Memory Efficient Vision Transformer with Cascaded Group AttentionHierarchical vision Transformer tuned for measured throughput rather than FLOPs. Each block puts one memory-bound attention layer between FFN layers. Cascaded group attention gives each head a channel split plus the previous head's output. Author-reported: EfficientViT-M5 gets 77.1% ImageNet-1K top-1 at 10,621 images/s on a V100, versus 75.2% and 7,560 for MobileNetV3-Large.Paper · 2023-05-11Exploiting Diffusion Prior for Real-World Image Super-ResolutionStableSR restores real-world low-resolution images with a frozen Stable Diffusion 2.1 by training only a time-aware encoder and scale-and-shift modulation layers. It adds an optional fidelity module (weight w) and Gaussian patch aggregation for arbitrary sizes. Author-reported: best FID (24.44) and CLIP-IQA/MUSIQ on DIV2K-Valid; lower PSNR, SSIM and LPIPS than several baselines.Paper · 2023-05-11InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningInstruction-tunes the BLIP-2 vision-language model on 13 held-in datasets, with frozen ViT and LLM. Only the Q-Former trains, and it also reads the instruction so its visual features depend on the task. Datasets are sampled by square root of size. Author-reported: new zero-shot results on all 13 held-out datasets, and 90.7% on ScienceQA (image context) after fine-tuning.Paper · 2023-05-10Benchmarking large language models for biomedical natural language processing applications and recommendationsBenchmarks GPT-3.5, GPT-4, LLaMA 2 13B and PMC LLaMA 13B on 12 biomedical NLP datasets under zero-shot, static and nearest-neighbour few-shot, and LoRA fine-tuning. Supervised BERT/BART baselines win most tasks; GPT-4 leads on medical question answering. Author-reported macro-average: 0.6536 (baselines) vs 0.4862 (best few-shot LLM).Paper · 2023-05-09StarCoder: may the source be with you!StarCoderBase is a 15.5B decoder-only code LLM (8K context, multi-query attention, fill-in-the-middle) trained on 1T tokens of permissively licensed code from The Stack. StarCoder fine-tunes it on 35B Python tokens. Author-reported: StarCoder reaches 33.6 pass@1 on HumanEval and 52.7 on MBPP.Paper · 2023-05-08Revisiting Relation Extraction in the era of Large Language ModelsFrames relation extraction as text generation. Few-shot GPT-3 nearly matches fully supervised models once outputs are re-judged by humans. Flan-T5 Large fine-tuned on GPT-3-written explanations is reported as SOTA on CoNLL04, ADE and NYT (micro-F1 80.76, 92.17, 95.23).Paper · 2023-05-07Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought PromptingCoT explanations can omit the real reason for an answer. Biasing features in the prompt (every few-shot answer labelled (A), or a suggested wrong answer) shift GPT-3.5 and Claude 1.0 answers while the explanation never mentions them. On 13 BIG-Bench Hard tasks accuracy falls by up to 36.3% (zero-shot, Suggested Answer, GPT-3.5); on BBQ the explanations support stereotype-aligned answers.Paper · 2023-05-07Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A TutorialTutorial on uncertainty quantification (UQ) for neural networks: Gaussian process regression, MC dropout, Gaussian-head neural network ensembles and spectral-normalized neural GPs (SNGP). Compares them on battery life and turbofan RUL prediction with RMSE, NLL and calibration. Author-reported: the ensemble has the lowest NLL on each battery test set; SNGP is often underconfident.Paper · 2023-05-05Otter: A Multi-Modal Model with In-Context Instruction TuningOtter adds in-context image and instruction-response examples to visual instruction tuning on OpenFlamingo-9B, training only the perceiver resampler and gated cross-attention layers. Trained on MIMIC-IT (about 3M pairs). Author-reported: Otter-Image-9B tops several listed image benchmarks (ScienceQA 69.3); Otter-Video beats mPLUG-Owl-V on MSVD-QA and MSRVTT-QA.Paper · 2023-05-03A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIMEPerspective on the XAI tools SHAP and LIME for tabular data. Their feature rankings change with the classifier and with collinear features, shown on four classifiers in a UK Biobank myocardial-infarction cohort and an airline-satisfaction case study. Proposes NMR and MIP to test and adjust ranking stability. Author-reported; no new benchmark.Paper · 2023-05-03Can Large Language Models Be an Alternative to Human Evaluations?Asks an LLM the same instruction, sample and 5-point questions that were given to human raters, then parses a number from its free-text answer. On story generation, text-davinci-003 and ChatGPT rank human-written stories above GPT-2 stories as three English teachers do, while T0 and text-curie-001 show no clear preference. Adversarial-attack text is scored too. Author-reported.Paper · 2023-05-02Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image GenerationCollects a public dataset of real users' prompts and pairwise picks (or ties) over two generated images, then trains PickScore, a CLIP ViT-H/14 scorer fine-tuned with a two-way softmax preference loss. PickScore predicts held-out user choices at 70.5% accuracy against 68.0% for expert annotators, and correlates better with real-user Elo than CLIP-H, ImageReward or HPS (author-reported).Paper · 2023-04-28Dissecting Recall of Factual Associations in Auto-Regressive Language ModelsTraces how GPT-2 XL and GPT-J answer subject-relation queries. Attention knockouts show two information stages: relation then subject. Early MLP layers enrich the subject representation, and upper attention heads extract the attribute, for 68.2% of GPT-2 XL queries (author-reported). Code covers GPT-2 XL only.Paper · 2023-04-27DataComp: In search of the next generation of multimodal datasetsDataComp is a benchmark for choosing image-text training data. Participants filter a 12.8B-pair Common Crawl pool or bring their own data, then train a fixed CLIP recipe at four scales and test on 38 tasks. Their best baseline, DataComp-1B (1.4B pairs), reaches 79.2% ImageNet zero-shot with ViT-L/14, 3.7 points above OpenAI's CLIP. Author-reported.Paper · 2023-04-27Proportionally Representative ClusteringProportionally representative fairness (PRF) is an axiom for centroid clustering: any sufficiently large group of agents must have a number of nearby centers proportional to its size. SEAR, a spatial expanding-approval rule, always returns a PRF outcome in polynomial time, as a 3-approximation to Proportional Fairness (PF) unconstrained and a (1+√2) one discrete. Theory only.Paper · 2023-04-27mPLUG-Owl: Modularization Empowers Large Language Models with MultimodalitymPLUG-Owl gives LLaMA-7B image input through a CLIP ViT-L/14 encoder and a learnable visual abstractor. Stage 1 aligns images with captions while the LLM is frozen; stage 2 trains LoRA on the LLM with text-only and image-text instructions. On the authors' 82-question OwlEval set, 66 answers are rated A or B against 54 for MiniGPT-4 (author-reported, manual rating).Paper · 2023-04-25Bake off redux: a review and experimental evaluation of recent time series classification algorithmsRe-runs the time series classification bake off: the best of eight algorithm families against 1-NN DTW on 112 UCR univariate problems, 30 resamples each, plus 30 new datasets. Author-reported: HIVE-COTE 2.0 (HC2) and MultiROCKET-Hydra are the most accurate (ACC 0.891 and 0.884 on 112 problems); QUANT trains fastest.Paper · 2023-04-22LLM+P: Empowering Large Language Models with Optimal Planning ProficiencyLLM+P has GPT-4 rewrite a natural-language planning problem as PDDL, guided by one (description, PDDL) example, then solves it with a classical planner (Fast Downward) for an optimal plan. Author-reported optimal-alias success over 7 IPC-derived domains: Blocksworld 90%, Grippers 95%, Storage 85%, Floortile 0%. GPT-4 writing plans directly reaches at most 35% in any domain.Paper · 2023-04-21DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-CorrectionPrompts GPT-4 or CodeX with four chained prompt modules for text-to-SQL, with no fine-tuning: schema linking, classifying the question as easy, non-nested or nested, class-specific few-shot SQL generation (NatSQL for non-nested), then zero-shot self-correction. Author-reported: 85.3% execution accuracy on the Spider holdout test with GPT-4, and 55.9% on the BIRD test.Paper · 2023-04-20GPT-NER: Named Entity Recognition via Large Language ModelsGPT-NER recasts named entity recognition as text generation: the LLM copies the sentence and wraps each entity in @@ and ## markers. Demonstrations are retrieved by token-level embedding similarity, and a yes/no self-verification pass removes false entities. With GPT-3 davinci-003 (author-reported) it reaches 90.91 F1 on full CoNLL2003 (BERT-MRC 93.04) and 82.20 on OntoNotes5.0 (BERT-MRC 91.11).Paper · 2023-04-20MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language ModelsAligns a frozen BLIP-2 vision encoder and Q-Former with a frozen Vicuna LLM through one trainable linear projection. Stage 1 trains it on about 5M image-caption pairs; stage 2 fine-tunes on about 3,500 curated detailed descriptions to fix unnatural output. Author-reported: 65/100 human-judged successes on four advanced tasks vs 5/100 for BLIP-2; 66.2% caption coverage vs 27.5%.Paper · 2023-04-20Segment Anything Model for Medical Image Analysis: an Experimental StudyEvaluates Meta's Segment Anything Model zero-shot on 19 medical imaging datasets (28 segmentation tasks) under five prompt modes. Box prompts beat points: one box per part averages 0.6542 IoU, and single prompts range from 0.1136 to 0.9118 across tasks. SAM beats three interactive baselines with one point but gains little from extra points. Author-reported.Paper · 2023-04-19Chameleon: Plug-and-Play Compositional Reasoning with Large Language ModelsChameleon wraps a frozen LLM with a module inventory (LLM prompts, image captioner, OCR, Bing search, Python program tools, rule-based table lookups). An LLM planner, prompted with few examples and no training, writes a short module sequence that runs in order on cached context. With GPT-4 it reports 86.54% on ScienceQA and 98.78% on TabMWP (author-reported).Paper · 2023-04-17Long-term Forecasting with TiDE: Time-series Dense EncoderTiDE is an MLP encoder-decoder for long-horizon forecasting: residual MLP blocks encode the look-back and covariates, decode the whole horizon, and a linear shortcut adds the history. Author-reported: lower Traffic MSE than PatchTST/64 at all four horizons, 5-10x faster than the best Transformer, and a near-optimal linear analogue for linear dynamical systems.Paper · 2023-04-17Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive CodingUses GPT-3 (davinci-text-002, temperature 0) with expert-written codebooks to label 668 French children's curiosity questions by complexity and syntactic structure. Agreement with experts is Cohen's kappa 0.61 (complexity, substantial) and 0.38 (syntax, fair). Codebook-style prompts beat example-style ones, and one example per code gives most of the gain. Preliminary, author-reported.Paper · 2023-04-17Synthetic Data from Diffusion Models Improves ImageNet ClassificationFine-tunes the Imagen text-to-image cascade on ImageNet-1K, prompting with class names, and samples synthetic training images. Author-reported: FID 1.76 and IS 239 at 256x256, CAS top-1 64.96 (256) and 69.24 (1024). Adding 1.2M synthetic images to real data raises ResNet-50 top-1 from 76.39 to 78.17.Paper · 2023-04-17Visual Instruction TuningLLaVA joins a frozen CLIP ViT-L/14 to the Vicuna LLM through one trainable linear projection. It is trained in two stages: image-caption alignment on CC3M, then instruction tuning on 158K GPT-4-written visual conversations. Author-reported: 85.1% relative score against text-only GPT-4 on LLaVA-Bench (COCO), and 92.53% on ScienceQA when GPT-4 judges between LLaVA and its own answers.Paper · 2023-04-14OpenAssistant Conversations -- Democratizing Large Language Model AlignmentReleases OpenAssistant Conversations, a corpus of 161,443 volunteer-written chat messages in 35 languages, with 461,292 quality ratings and reply rankings, collected on a public web platform. Several rankers' orders are merged with ranked pairs. Pythia, Falcon and LLaMA models tuned on it report gains over their base models on most reported benchmarks (author-reported).Paper · 2023-04-14Very high resolution canopy height maps from RGB imagery using self-supervised vision transformer and convolutional decoder trained on Aerial LidarCanopy height maps from RGB satellite imagery: a DINOv2 ViT-H pretrained on 18M unlabeled Maxar tiles, a frozen encoder with a DPT decoder trained on NEON aerial lidar (256-bin classification, Sigloss), then GEDI-based per-pixel rescaling. Author-reported average MAE 2.8 m over three lidar test sets.Paper · 2023-04-12ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual LearningZero-shot evaluation of ChatGPT on seven NLP tasks (POS tagging, NER, relation extraction, NLI, QA, commonsense reasoning, summarization) across 34 to 37 languages, with one prompt per example: a task description, the input and a format note. ChatGPT trails supervised models on most tasks; POS tagging is the exception, and English task descriptions usually score higher. Author-reported.Paper · 2023-04-12DreamPose: Fashion Image-to-Video Synthesis via Stable DiffusionDreamPose animates one fashion photo along a DensePose pose sequence. It fine-tunes Stable Diffusion on UBC Fashion videos, adds CLIP and VAE image tokens and stacked pose maps to the UNet, then fine-tunes per subject. Author-reported: FID 13.04 and FVD 238.75 on UBC Fashion, ahead of MRAA, TPSMM and PIDM.Paper · 2023-04-12ImageReward: Learning and Evaluating Human Preferences for Text-to-Image GenerationTrains ImageReward, a BLIP-based reward model, on 136,892 expert pairwise image rankings, and ReFL, which back-propagates the reward through one late denoising step of Stable Diffusion v1.4. Author-reported: 65.14% pairwise preference accuracy vs 57.76% for BLIP score; ReFL wins 58.79% of human comparisons against the baseline.Paper · 2023-04-12UniverSeg: Universal Medical Image SegmentationUniverSeg segments an unseen medical task with no training: a fully convolutional network takes a query image and a support set of image-label pairs, and CrossBlocks exchange information between them. Trained on MegaMedical (53 datasets) with task augmentation and synthetic tasks. Author-reported mean Dice 71.8 on six unseen held-out datasets, vs 50.1 for SE-net and 84.4 for per-task nnUNet.Paper · 2023-04-11SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports ScenesIntroduces SportsMOT, 240 basketball, volleyball and football tracking videos (about 150K frames, 1.6M boxes) where only the on-court players are tracked, and MixSort, which adds a MixFormer-based appearance similarity to tracking-by-detection trackers. Author-reported: MixSort-OC reaches 74.1 HOTA on the SportsMOT test set, MixSort-Byte 65.7 vs 64.1 for ByteTrack.Paper · 2023-04-11Teaching Large Language Models to Self-DebugSelf-Debugging teaches a frozen LLM to debug its own code with few-shot prompts: run the program, explain it line by line, judge it from the run or the explanation, and fix it, for up to 10 turns. Author-reported: Codex on Spider 81.3 to 84.1; on TransCoder with unit tests 80.4 to 92.5; GPT-4 on MBPP 72.8 to 80.6.Paper · 2023-04-06A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluationBenchmarks 9 augmentation methods (SMOTE variants, ADASYN, RUS, ROS, CT-GAN) with 9 ensemble models on 23 binary and 4 Abalone-derived multi-class imbalanced sets, 30 runs each. The conclusion names SMOTE + LightGBM and ROS + LightGBM as best and says plain SMOTE and ROS beat CT-GAN at lower cost. Author-reported.Paper · 2023-04-06RFAConv: Receptive-Field Attention Convolution for Improving Convolutional Neural NetworksRFAConv replaces a k×k convolution with one whose weights differ per position: each k×k receptive-field window gets its own softmax attention over its k×k slots, built from average pooling and grouped convolutions. On ImageNet-1k, ResNet18 top-1 rises from 69.59 to 71.23 for about 0.16M extra parameters (author-reported).Paper · 2023-04-06Training-Free Layout Control with Cross-Attention GuidanceSteers a pre-trained Stable Diffusion model to a user-given layout without training: during the first denoising steps, the latent is updated by gradients of an energy that pulls each prompt word's cross-attention mass into its bounding box ("backward guidance"). Compared with biasing attention directly ("forward guidance"), it reaches 95.95% conditional VISOR accuracy (author-reported).Paper · 2023-04-04LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language ModelsCompares four parameter-efficient fine-tuning families (prefix-tuning, series and parallel adapters, LoRA) on LLaMA-7B/13B, BLOOMz-7B and GPT-J-6B, trained on 10K math and 170K commonsense samples. Author-reported: LLaMA-13B with a parallel adapter averages 81.5 on eight commonsense tasks versus 77.0 for ChatGPT. Its math wins rest on benchmarks that overlap the training data.Paper · 2023-04-04Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language ModelsSurvey of early ChatGPT research: a bibliometric pass over 194 arXiv papers mentioning ChatGPT (as of 1 April 2023) with a submission trend, word cloud and category counts, then narrative summaries of applications (QA, classification, generation, code, extraction, medicine), ethics, third-party evaluations, limitations and future directions. No new experiments.Paper · 2023-04-04The Vector Grounding ProblemA position paper arguing that LLMs can have internal states that refer to the world, so their outputs can carry meaning independent of the reader. Referential grounding needs a causal link to the world plus a selection history, which preference tuning supplies most clearly. Pre-training and in-context learning are argued as speculative routes. No experiments, so no measured results.Paper · 2023-04-03DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean TaskFine-tunes ChatGLM-6B on Chinese medical Q&A dialogues with LoRA (rank 8, alpha 16, query and value projections), one epoch, Lion at 2e-5. Translates ChatDoctor into Chinese with a model distilled from ChatGPT translations. No quantitative benchmark: doctor-annotated example answers only. Author-reported: one A100 for 8 to 13 hours, about 18.75 USD per 100K QA pairs.Paper · 2023-04-03Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingPythia is a suite of 8 decoder-only LMs (70M to 12B), each trained twice on the Pile and the deduplicated Pile in the same data order, with 154 checkpoints per model. Three case studies (gender-pronoun retraining, memorization and training order, term frequency) use it. Author-reported: retraining on feminine pronouns lowers measured gender bias.Paper · 2023-04-02A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NASA survey of the YOLO real-time object detectors, from YOLOv1 to YOLOv8, YOLO-NAS and YOLO with Transformers. It covers AP and NMS, then each version's architecture, training tricks and reported COCO AP, with a summary table. Several AP values differ between the text and the table.Paper · 2023-03-31CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model SocietyTwo chat LLM agents, an assistant and a user, role-play a task that a task-specifier agent first makes concrete. Inception system prompts fix roles, one instruction per turn and a <CAMEL_TASK_DONE> stop token. Author-reported: CAMEL solutions beat single-shot gpt-3.5-turbo (76.3% human wins on AI Society, 76.0% GPT-4 wins on Code); fine-tuned LLaMA-7B reaches 14.0 HumanEval pass@1.Paper · 2023-03-30Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical StudyProbes ChatGPT with 24 Hofstede Culture Survey questions, asked in English with a country named, in that country's language, and in a third English wording. Six dimension scores from a weighted-difference formula are correlated with human scores (Spearman). Author-reported: alignment is strongest for the US, and English prompts shrink the spread across cultures.Paper · 2023-03-30CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-XCodeGeeX is a 13B decoder-only code model pretrained on 850B tokens in 23 languages, plus HumanEval-X, a 164-problem benchmark hand-translated into C++, Java, JavaScript and Go. Author-reported average pass@1 on HumanEval-X generation: 18.40%, against 16.73% for CodeGen-Multi-16B. INT8 weights cut inference memory from about 26.9 GB to 14.7 GB.Paper · 2023-03-30Iterative Prompt Learning for Unsupervised Backlit Image EnhancementCLIP-LIT enhances backlit photos without paired data. It learns a positive/negative prompt pair in CLIP space, trains a network that predicts an illumination map (output = input / map) with CLIP and identity losses, and alternates prompt refinement with network tuning. Author-reported: PSNR 21.58 and MUSIQ 55.68 on the BAID test set, vs 19.74 PSNR for the best released baseline (Zero-DCE).Paper · 2023-03-30LayoutDiffusion: Controllable Diffusion Model for Layout-to-image GenerationLayoutDiffusion is an image-space diffusion model that takes boxes and classes as input. A layout transformer fuses the objects, and object-aware cross-attention lets each image patch attend to objects by box position. Trained with classifier-free dropout; sampled with DPM-Solver in 25 steps. Author-reported: COCO-Stuff 256 FID 15.61 vs 29.10 (PLGAN); VG 256 FID 15.63.Paper · 2023-03-30Self-Refine: Iterative Refinement with Self-FeedbackSelf-Refine loops one LLM through three few-shot prompts: generate a draft, critique it with specific, actionable feedback, rewrite it using that feedback plus the history, until a stop signal or 4 iterations. No training. On 7 tasks with GPT-3.5, ChatGPT and GPT-4 it beats one-shot generation, e.g. GPT-4 code optimization 27.3% to 36.0%; math gains are near zero.Paper · 2023-03-29VideoMAE V2: Scaling Video Masked Autoencoders with Dual MaskingScales masked video autoencoding (VideoMAE) to a 1B-parameter ViT-g. Dual masking hides 90% of encoder tokens and lets the decoder reconstruct only half of the cubes, which cuts compute and memory. Pre-trains on 1.35M unlabeled clips, then post-pre-trains on a labeled Kinetics mix. Author-reported: 90.0% top-1 on Kinetics-400 and 77.0% on Something-Something V2.Paper · 2023-03-27Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning PerspectiveLIQE fine-tunes CLIP ViT-B/32 to score image quality without a reference. Each image is matched to 495 joint text labels (5 quality levels x 9 scenes x 11 distortions); the marginals give a quality score plus scene and distortion heads, trained with fidelity losses and dynamic loss weights. Author-reported median SRCC 0.970 on LIVE, 0.930 on KADID-10k and 0.919 on KonIQ-10k over ten splits.Paper · 2023-03-27ChatGPT Outperforms Crowd-Workers for Text-Annotation TasksZero-shot ChatGPT (gpt-3.5-turbo) labels five text-annotation tasks on four tweet and news samples, compared with MTurk crowd-workers and trained annotators. Accuracy against trained annotators is about 25 points above MTurk on average, ChatGPT's intercoder agreement is highest (97% at temperature 0.2), and cost is under $0.003 per annotation. Author-reported.Paper · 2023-03-27Sigmoid Loss for Language Image Pre-TrainingSigLIP replaces CLIP's batch-softmax contrastive loss with an independent binary sigmoid loss on every image-text pair, plus a learnable bias initialised at -10. No global normalisation is needed, so the loss can be chunked across devices. It beats softmax below 16k batch, saturates near 32k, and a SigLiT model reaches 84.5% ImageNet zero-shot on four TPUv4 chips in two days.Paper · 2023-03-27The Stable Signature: Rooting Watermarks in Latent Diffusion ModelsFine-tunes only the decoder of a latent diffusion model so every output carries a fixed 48-bit signature, read by a frozen HiDDeN-style extractor. A binomial test on matching bits flags generated images; per-user keys allow identification. Author-reported: 0.99 bit accuracy on unmodified text-to-image output, 0.92 after a combined crop, brightness and JPEG edit; FID changes by at most 0.6.Paper · 2023-03-25Human Preference Score: Better Aligning Text-to-Image Models with Human PreferenceHuman Preference Score (HPS) fine-tunes CLIP ViT-L/14 on human choices among Stable Diffusion images to rank images by preference. Author-reported: HPS predicts choices better than CLIP (43.5% vs 32.9%). A LoRA adapter for Stable Diffusion v1.4, trained with HPS-selected images and a negative-prompt marker, is preferred over the original in a 100-prompt user study.Paper · 2023-03-25Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A SurveyReview of graph-plus-time neural forecasting models (STGNNs) for city data such as traffic, air quality and crime. Covers graph construction, the basic GNN, RNN, TCN and attention blocks with their fusion, recent variants (multi-graph, adaptive graph, multi-scale, decomposition, NAS) and advanced frameworks. No experiments. Headline stat: transportation is over 60% of the surveyed work.Paper · 2023-03-24CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic ImagesGenerates 60,000 Stable Diffusion v1.4 images that mirror the ten CIFAR-10 classes, then trains a small CNN (two 32-filter conv layers, one 64-unit dense layer) to label real versus generated images. Author-reported: 93.55% accuracy and F1 0.936 for the final model. Grad-CAM heatmaps suggest the model relies on background artefacts rather than the main subject.Paper · 2023-03-24ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain KnowledgeFine-tunes LLaMA-7B on about 100k real patient-doctor dialogues from HealthCareMagic, then adds an autonomous retrieval loop over Wikipedia or a disease database: the model writes keywords, ranks passages by keyword hits, and answers from the best ones. On 10k iCliniq questions, BERTScore F1 is 0.8446 for ChatDoctor vs 0.8406 for ChatGPT (author-reported).Paper · 2023-03-24Knowledge Graphs: Opportunities and ChallengesSurvey of knowledge graphs. Opportunities: how KGs help recommender, question-answering and retrieval systems, plus uses in education, science, social networks and medicine. Challenges: limits of five core technologies (embeddings, acquisition, completion, fusion, reasoning). Proposes no new method; tabulates published Hits@10 on FB15K for about 20 embedding models.Paper · 2023-03-24Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion PriorTwo-stage single-image 3D creation. Stage 1 fits a NeRF with Stable Diffusion score distillation at novel views, a CLIP-based diffusion loss near the reference view and a depth prior. Stage 2 colours a point cloud from the photo and refines hidden texture with a learned neural renderer. Author-reported DTU LPIPS 0.0908 (enhanced) vs 0.4075 for DreamFusion+.Paper · 2023-03-23ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language ModelReview article on using ChatGPT-style multi-modal LLMs in dentistry: text mining and report generation from records, synthetic quasi-EHR data, vision-language and audio-language diagnosis, and an illustrative end-to-end pipeline. It reports no quantitative evaluation; the example outputs are illustrative only.Paper · 2023-03-23TinyML: Tools, Applications, Challenges, and Future Research DirectionsA survey of TinyML, machine learning on cheap, power-constrained microcontrollers. It lists tool-chains and hardware platforms, reviews applications (speech, vision, health, edge, BCI, vehicles, ecology, anomaly detection, predictive maintenance) and lists open challenges. It proposes no method and reports no experiments of its own.Paper · 2023-03-22Sparks of Artificial General Intelligence: Early experiments with GPT-4Qualitative and small-scale quantitative probing of an early, text-only GPT-4 with novel tasks across coding, math, tools, theory of mind and more. Author-reported: 82% HumanEval pass@1, 38/53% pass@1/5 on 100 post-cutoff LeetCode problems (humans 38.2%), 87.1% GSM8K, 42.5% MATH, plus systematic failures in planning and multi-step arithmetic.Paper · 2023-03-21ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly PublishingPosition paper (no experiments) on ChatGPT in academic publishing. Gives a non-technical primer on GPT and ChatGPT, lists possible benefits (editing, summaries, citation help, translation) and ethical risks (bias, authorship, copyright, plagiarism, missing references, Matthew effect, publish-or-perish), and sketches an untested AI-text detection pipeline and tenure reform.Paper · 2023-03-21Sample4Geo: Hard Negative Sampling For Cross-View Geo-LocalisationMatches street-view or drone images to satellite tiles with one shared ConvNeXt-B encoder trained by a symmetric InfoNCE loss, without polar transforms or aggregation modules. Hard negatives come from GPS-nearby locations early and from cosine similarity later (Dynamic Similarity Sampling). Author-reported state-of-the-art R@1 on CVUSA, CVACT, University-1652 and VIGOR.Paper · 2023-03-21TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question AnsweringTIFA scores whether a generated image matches its prompt, without references. A language model turns the prompt into yes/no and multiple-choice questions, UnifiedQA filters them, and a VQA model answers them on the image; the score is the share answered correctly. Author-reported Spearman 0.597 with human ratings for mPLUG, vs 0.332 for CLIPScore. Releases TIFA v1.0: 4,081 prompts.Paper · 2023-03-21Text2Motion: From Natural Language Instructions to Feasible PlansText2Motion has an LLM propose skill sequences over learned Pick, Place, Pull and Push skills. Each candidate is checked by geometric feasibility planning (STAP) and Q-ensemble OOD rejection, and a hybrid of shooting and greedy search picks the plan. Author-reported: 82% success on a simulated tabletop suite, vs 13% for prior language planners.Paper · 2023-03-20DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4Zero-shot removal of identifiers from clinical notes: a prompt built from HIPAA-style PHI categories asks ChatGPT or GPT-4 to replace them with "[redacted]". Author-reported on the 2014 i2b2/UTHealth set: GPT-4 with an explicit prompt reaches 0.99 entity accuracy (on 50 hand-run samples), ChatGPT 0.929, fine-tuned ClinicalBERT 0.974.Paper · 2023-03-20EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction ReasoningEqMotion predicts the future motion of interacting agents with a network that is equivariant to rotation, reflection and translation. Coordinates are updated by linear, attention and gated operations, while interaction categories and pattern features come from invariant quantities. Author-reported state of the art on particle, molecule, H3.6M skeleton and ETH-UCY pedestrian prediction.Paper · 2023-03-20Reflexion: Language Agents with Verbal Reinforcement LearningReflexion improves LLM agents across retries without updating weights. After each failed trial the model writes a verbal self-reflection, stored in a bounded memory and added to the next prompt. Reports 91.0 pass@1 on HumanEval Python (GPT-4 baseline 80.1), +22% absolute on ALFWorld over ReAct, and +20% on HotPotQA. All results author-reported.Paper · 2023-03-17Can AI-Generated Text be Reliably Detected?Stress-tests AI-text detectors (watermark, trained, zero-shot, retrieval). Recursive paraphrasing with DIPPER or LLaMA-2 drops watermark TPR@1%FPR from 99.8% to under 20% by round 5 at a small perplexity cost. Spoofing learned green lists drops watermark AUROC from 99.8% to 1.3%. A theorem bounds detector AUROC by the total variation distance between human and AI text.Paper · 2023-03-17Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping ReviewSystematic scoping review of 118 peer-reviewed studies (2017-2022) that use language models to automate educational tasks. It maps 53 use cases in nine categories and finds low technology readiness, poor replicability (107 of 118 studies not replicable from the paper), low transparency (109 at Tier 1) and gaps in privacy, equality and beneficence. Author-reported counts; no model is trained.Paper · 2023-03-16A Short Survey of Viewing Large Language Models in Legal AspectCritical narrative survey proposing the authority-grounded legal AI system: LLM legal tools are judged on task capability, mechanisms (adaptation, RAG, knowledge graphs, agents) and assurance evidence (source status, jurisdiction, time, proposition support). A structured evidence map of 327 technical studies from 2022-2025 supports it. The framework is a proposal, not a validated standard.Paper · 2023-03-16DiffIR: Efficient Diffusion Model for Image RestorationDiffIR runs a diffusion model on a compact 256-value restoration prior (IPR), not on pixels. A pretrained encoder gives the prior from ground truth; a diffusion model predicts it from the low-quality image in 4 steps, and the restorer is jointly trained. Author-reported SOTA on inpainting, x4 SR and deblurring at lower compute than earlier diffusion restorers.Paper · 2023-03-14LayoutDM: Discrete Diffusion Model for Controllable Layout GenerationLayoutDM generates layouts (labelled bounding boxes) with a discrete diffusion model over quantized tokens, one corruption matrix per attribute and a [PAD] token for variable element counts. Known fields, refinement and relation constraints are injected at inference by masking and logit adjustment, without retraining. Author-reported: unconditional Rico FID 6.65.Paper · 2023-03-14V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative PerceptionPresents V2V4Real, a real-world dataset of two cars driving together in Columbus, Ohio: 410 km, 20K LiDAR frames, 40K RGB frames, 240K 3D boxes for 5 vehicle classes, and HD maps. It defines benchmarks for cooperative 3D detection, tracking and Sim2Real domain adaptation. Author-reported: CoBEVT reaches 66.5 AP@0.5 (sync) against 39.8 for no fusion.Paper · 2023-03-13FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency RegularizationFreeNeRF adds two training-time regularizers to a plain NeRF for few-shot novel view synthesis: a frequency mask that reveals positional-encoding bands gradually, and a penalty on density near the camera. Both are cheap, with no pre-training or patch rendering. The authors report state-of-the-art results on Blender, DTU and LLFF.Paper · 2023-03-13NeuSE: Neural SE(3)-Equivariant Embedding for Long-Term Object-Based Simultaneous Localization and MappingEncodes each object from a partial point cloud as a latent point cloud that transforms with the object's SE(3) pose and scale, so relative poses come from the codes, not render-and-optimize loops. The codes also drive data association, pose constraints and change-aware mapping. On a real 4-Round sequence, ORB-SLAM3's full ATE drops from 0.128 m to 0.093 m (author-reported).Paper · 2023-03-13RTMPose: Real-Time Multi-Person Pose Estimation based on MMPoseRTMPose is a top-down 2D pose estimator. It predicts each keypoint as two 1-D SimCC bin distributions, using a CSPNeXt backbone, a 7x7 convolution and one gated attention unit, trained with strong-then-weak augmentation. Author-reported: 75.8 AP on COCO (with flip test) for RTMPose-m and 72.2 AP for RTMPose-s, which runs at 70+ FPS on a Snapdragon 865.Paper · 2023-03-10TSMixer: An All-MLP Architecture for Time Series ForecastingTSMixer is an all-MLP forecaster. It alternates a time-mixing layer, shared across variates, with a feature-mixing MLP, shared across time, using residuals and normalization, then projects input length to horizon. Author-reported: comparable to PatchTST on seven long-term benchmarks; lowest M5 test WRMSSE in each setting (0.737 without, 0.640 with auxiliary features).Paper · 2023-03-07Larger language models do in-context learning differentlyTests whether in-context learning relies on semantic priors or on input-label mappings in the prompt. With flipped labels, large models override priors (emergent with scale); with semantically unrelated labels (foo/bar), large models still learn the mapping. Instruction tuning strengthens both. Largest Codex model beats chance on 16-dimensional linear classification by 19 points (author-reported).Paper · 2023-03-07Run, Don't Walk: Chasing Higher FLOPS for Faster Neural NetworksPartial convolution (PConv) runs a regular 3x3 conv on only 1/4 of the channels and passes the rest through, followed by pointwise convs. That gives 1/16 the FLOPs of a full conv but far higher FLOPS than depthwise conv. FasterNet, built from it, reaches 71.9% (T0) to 83.5% (L) ImageNet top-1 and beats MobileViT, Swin and ConvNeXt on GPU, CPU and ARM speed.Paper · 2023-03-06Real-World Humanoid Locomotion with Reinforcement LearningWalking policy for the full-size Agility Digit humanoid, trained only in simulation: a causal transformer reads the history of observations and actions and predicts the next action, trained with PPO plus a KL term to a state-based teacher. Zero-shot outdoor deployment with no falls reported over a week of testing.Paper · 2023-03-02Human Motion Diffusion as a Generative PriorReuses one pretrained text-to-motion diffusion model (MDM) for three compositions: DoubleTake joins prompted intervals into long motions without retraining, ComMDM adds a small block for two-person motion, DiffusionBlending mixes fine-tuned control models. Author-reported: DoubleTake HumanML3D motion FID 0.60; BABEL transition FID 1.88 vs 3.86 for TEACH.Paper · 2023-03-01Efficient and Explicit Modelling of Image Hierarchies for Image RestorationGRL is an image-restoration transformer whose layers run three branches in parallel: window attention for the regional range, anchored stripe attention for long-range dependencies through pooled anchors, and a channel-attention convolution for local detail. Author-reported state of the art on several of its 7 tasks, e.g. 33.93 dB on GoPro deblurring against Restormer's 32.92 dB.Paper · 2023-02-28Large Language Models Are State-of-the-Art Evaluators of Translation QualityGEMBA scores each source-translation pair with a zero-shot prompt to a GPT model (0-100 scale, 1-5 stars or a quality class), with or without a human reference, then averages per system. Author-reported: GPT-4 with a direct-assessment prompt and a reference reaches 89.8% system-level pairwise accuracy on WMT22 MQM, above every metric in its main table, including non-public MetricX XXL (85.0%).Paper · 2023-02-27LLaMA: Open and Efficient Foundation Language ModelsDecoder-only transformers from 7B to 65B parameters, trained on 1.0T to 1.4T tokens of publicly available text. Pre-norm RMSNorm, SwiGLU and rotary embeddings, no absolute positions. Author-reported: LLaMA-13B beats GPT-3 (175B) on most common-sense benchmarks; LLaMA-65B is close to Chinchilla-70B and PaLM-540B.Paper · 2023-02-24Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated FeedbackLLM-Augmenter wraps a frozen black-box LLM (ChatGPT) with plug-and-play modules: a Knowledge Consolidator chains retrieved evidence, and a Utility module scores each candidate and feeds back notes that re-prompt the LLM until the answer passes. Knowledge F1 gains about 10 points on news chat and 6 on customer service over ChatGPT alone (author-reported).Paper · 2023-02-24Language-Driven Representation Learning for RoboticsVoltron pretrains a ViT on Something-Something-v2 video-caption pairs. It masks frames and reconstructs them while conditioning on or generating the caption, with a tunable mix alpha. Author-reported: beats reimplemented MVP and R3M on four quantitative robot tasks, e.g. referring-expression grounding 89.4 vs 63.3 (R-R3M) and 49.6 (R-MVP).Paper · 2023-02-23Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?InfoSeek is a visual QA benchmark of information-seeking questions: 1.35M generated Wikidata items plus 8.9K human-written ones. It tests end-to-end models (No-KB) and a CLIP entity-retrieval plus Wikipedia-reader pipeline (With-KB). Author-reported: best No-KB PaLI-X gets 22.1 overall on Wikidata; the FiD pipeline gets 18.2 on the human split, against 10.8 for the best No-KB model.Paper · 2023-02-23Side Adapter Network for Open-Vocabulary Semantic SegmentationSAN attaches a light side ViT (8 blocks, width 240, 8.4M trainable parameters) to a frozen CLIP. Its mask branch predicts mask proposals, and its attention-bias branch steers CLIP's last blocks to classify each mask. Trained end-to-end on COCO-Stuff, it reaches 27.5 mIoU on ADE-150 with ViT-B/16 and 32.1 with ViT-L/14. Author-reported.Paper · 2023-02-21ChatGPT: Jack of all trades, master of noneTests ChatGPT (version not stated) on 25 analytical NLP tasks, mostly subjective, with prompts and manual answer extraction. Averaged over tasks, it scores 56.5 against 73.7 for the best reference system, a 25.5% loss; emotion tasks lose most. Adds personalized in-context prompts, prompt-pattern and context tests, and a GPT-4 check on five tasks. Author-reported.Paper · 2023-02-21RealFusion: 360° Reconstruction of Any Object from a Single ImageRealFusion reconstructs a full 360° radiance field of an object from one masked RGBA photo. It fits an Instant-NGP model to the photo and adds score distillation from Stable Diffusion on random views, prompted by a token learned from augmented copies of the photo. Author-reported: mean CO3D F-score 9.58 vs 8.24 for Shelf-Supervised Mesh Prediction over 7 categories.Paper · 2023-02-20Composer: Creative and Controllable Image Synthesis with Composable ConditionsComposer splits each training image into eight representations (caption, CLIP embedding, color histogram, sketch, instance masks, depth, grayscale, masked image) and trains a cascaded diffusion model to rebuild the image from random subsets of them. Chosen conditions can then be recombined at inference for layout, palette, style and region control. Author-reported COCO FID 9.2.Paper · 2023-02-18How Good Are GPT Models at Machine Translation? A Comprehensive EvaluationEvaluates three OpenAI GPT models (text-davinci-002, text-davinci-003, ChatGPT) for machine translation in 18 directions, against WMT best systems and Microsoft Translator. Uses 0 to 5 shots, document-level windows and a hybrid that falls back to GPT. Text-davinci-003 is competitive on high-resource pairs, weaker on low-resource ones, and the hybrid scores highest. Author-reported.Paper · 2023-02-18MAILS -- Meta AI Literacy Scale: Development and Testing of an AI Literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change- and Meta-CompetenciesMAILS, a self-report AI literacy questionnaire: 34 items rated 0 to 10, covering four AI literacy facets, a separate Create AI factor and 12 AI self-management items. Confirmatory factor analysis on 300 German-speaking adults gives good fit for the modified model (CFI .926, RMSEA .057). The modifications were made after seeing the data, and no external validity was tested.Paper · 2023-02-16Auditing large language models: a three-layered approachProposes auditing LLMs at three linked levels: governance audits of the provider, model audits before release, and application audits of products built on the model. Claims audits must be external, collaborative and continuous after launch. The paper has no experiments; the blueprint is argued from a literature review and expert interviews, not measured.Paper · 2023-02-16Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based acceleratorsHardware-aware (HWA) training retrains 11 DNNs (CNNs, LSTMs, RNN-T, BERT and ALBERT) with a simulated analog in-memory crossbar in the forward pass: PCM programming error, drift, read and output noise, IR-drop and 8-bit converters. Five reach iso-accuracy (normalized accuracy above 99%) after 1 hour of drift. Author-reported.Paper · 2023-02-16Large Language Models Fail on Trivial Alterations to Theory-of-Mind TasksCritique with experiments, not a method: small edits to two published Theory-of-Mind vignettes that keep the ToM logic intact (unexpected contents, unexpected transfer) flip GPT-3.5's belief completions, e.g. 95% chocolate for a bag the reader can see into. Author-reported probabilities from one model. No code.Paper · 2023-02-16MultiDiffusion: Fusing Diffusion Paths for Controlled Image GenerationRuns one pretrained text-to-image diffusion model on overlapping crops (for wide panoramas) or on masked regions (for layouts) at every denoising step, then averages the crop outputs per pixel. No training or finetuning. Author-reported: panorama FID 10.3 vs 18.4 (BLD) and 45.5 (Stable Inpainting); region IoU 0.26 vs 0.17 (BLD) and 0.16 (SI) on a COCO subset.Paper · 2023-02-16T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion ModelsT2I-Adapter trains small adapter networks that read a control map (sketch, segmentation, keypose, depth, colour) and add multi-scale features to the encoder of a frozen Stable Diffusion UNet. Adapters can be composed with weights and run once per sample. Author-reported COCO val FID 16.78 with text plus segmentation, against 24.68 for Stable Diffusion with text only.Paper · 2023-02-14ConceptFusion: Open-set Multimodal 3D MappingConceptFusion stores CLIP-aligned features in a 3D map. Each pixel's feature mixes a whole-image embedding with per-mask region embeddings and is fused by SLAM-style point fusion. Text, image, click and audio queries score map points by cosine similarity, with no training. Author-reported: 0.446 3D mIoU on UnCoCo structured text queries, versus 0.289 for supervised OpenSeg-3D.Paper · 2023-02-08Is ChatGPT a General-Purpose Natural Language Processing Task Solver?Zero-shot evaluation of ChatGPT against GPT-3.5 (text-davinci-003) on 20 NLP datasets in 7 task categories. Each task gets one instruction-plus-input prompt; reasoning tasks add a two-stage chain-of-thought step. ChatGPT beats GPT-3.5 on most arithmetic, NLI, dialogue and sentiment sets, but trails on summarization, NER and several commonsense tasks, and stays well below fine-tuned models.Paper · 2023-02-06A Categorical Archive of ChatGPT FailuresQualitative archive of ChatGPT failures in eleven categories: reasoning, logic, arithmetic, facts, bias, humor, coding, grammar, self-awareness, ethics and other. Examples are mostly public posts plus the author's own tests. Gives no counts, no method and no quantitative results. Useful as a source of hard test cases.Paper · 2023-02-06Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering DesignA review and practical guide to metrics for deep generative models in engineering design. It groups metrics into similarity, novelty and diversity, constraints, performance and target achievement, and conditioning. On 2-D toys, similarity alone can rank models wrongly. Two case studies show the metrics. Author-reported; the repo holds the 2-D code only.Paper · 2023-02-06Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image RetrievalTrains a three-layer MLP that maps a frozen CLIP ViT-L/14 image embedding to one pseudo word token, using only unlabeled images and a contrastive loss. At query time the token fills a placeholder in a text prompt, and the prompt embedding is matched to candidate images. Author-reported: zero-shot CIRR test R@10 65.3 and Fashion-IQ validation R@10 24.7 (average).Paper · 2023-02-02Accelerating Large Language Model Decoding with Speculative SamplingSpeculative sampling (SpS): a small draft model proposes K tokens, the large target model scores all K+1 prefixes in one parallel call, and a modified rejection sampling rule accepts a prefix while exactly preserving the target distribution. On Chinchilla 70B with a 4B draft (K=4, 16 TPU v4) it gives 1.92-2.46x decoding speedup with unchanged benchmark scores.Paper · 2023-02-02Multimodal Chain-of-Thought Reasoning in Language ModelsTwo-stage chain-of-thought for image-and-text questions. A fine-tuned T5 writes a rationale from text and frozen ViT features, then answers from the text plus that rationale. Author-reported: 90.45% on ScienceQA test (738M model), above the prior best cited in its table (86.54%) but below concurrent LLaVA-13B (90.92%).Paper · 2023-02-01Analyzing Leakage of Personally Identifiable Information in Language ModelsMeasures PII leakage from GPT-2 fine-tuned on court cases, emails and health reviews, with black-box extraction, reconstruction and inference attacks. Reconstruction samples candidate names from the model and ranks them by sentence perplexity, using the suffix. Author-reported: up to 10x more PII than the TAB attack; DP (eps=8) still leaks about 3% of PII sequences.Paper · 2023-01-31Benchmarking Large Language Models for News SummarizationHuman ratings of ten LLMs on CNN/DM and XSUM. Instruction tuning, not model size, drives zero-shot summary quality, and the standard reference summaries are low quality. Freelance-writer summaries were rated on par with the best zero-shot instruct model, which copies far more from the article. Results are author-reported.Paper · 2023-01-31In-Context Retrieval-Augmented Language ModelsIn-Context RALM prepends retrieved passages to the input of a frozen, off-the-shelf language model, with no change to its architecture or weights. BM25 retrieval every 4 tokens with a 32-token query works best among the retrievers tested; zero-shot LM reranking and a trained reranker add further gains. Authors report gains equal to a 2-3x larger model on five corpora.Paper · 2023-01-31Large Language Models Can Be Easily Distracted by Irrelevant ContextAdds one irrelevant sentence to 100 GSM8K base problems (58,052 variants, GSM-IC) and measures prompting methods. Chain-of-thought, least-to-most and program prompts fall from 83-95% on the base problems to 62-81% micro accuracy. Self-consistency (20 samples) lifts least-to-most to 93.4% micro but only 45.0% macro. An instruction to ignore irrelevant information also helps. Author-reported.Paper · 2023-01-31Learning Universal Policies via Text-Guided Video GenerationUniPi plans by text-conditioned video diffusion: from the observed image and a text goal it generates future frames, and an inverse-dynamics model reads actions off them, executed open-loop. Author-reported: 59.1% and 53.2% task completion on seen place and relation instructions in simulated manipulation, versus at most 19.4% and 12.8% for baselines.Paper · 2023-01-31The Flan Collection: Designing Data and Methods for Effective Instruction TuningAblates the design choices behind the Flan 2022 instruction-tuning collection on T5-XL (3B): mixing zero- and few-shot prompt templates, scaling to 1.8k+ tasks, input inversion and mixture balancing. Mixed prompt training helps in both settings; the full recipe beats other T5-XL collections on most reported metrics (author-reported).Paper · 2023-01-30BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsBLIP-2 bridges a frozen image encoder and a frozen LLM with a small Querying Transformer (Q-Former), pre-trained in two stages: image-text representation learning, then vision-to-language generation through projected soft prompts. Author-reported: 65.0% zero-shot VQAv2 test-dev with ViT-g and FlanT5-XXL, 8.7 points above Flamingo-80B.Paper · 2023-01-30Specializing Smaller Language Models towards Multi-Step ReasoningDistills GSM8K chain-of-thought solutions from code-davinci-002 into T5 and FlanT5 models (250M to 11B) with a top-5 distribution-matching loss, and picks checkpoints on held-out math sets. Specialized FlanT5-XXL reaches 27.1% GSM8K accuracy (author-reported), but its BIG-Bench Hard chain-of-thought accuracy drops to 0.0.Paper · 2023-01-26DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability CurvatureDetectGPT flags a passage as model-written when the source LLM gives it a much higher log probability than it gives to small random rewrites of it. The rewrites come from T5 mask-filling; no classifier is trained. Author-reported AUROC on XSum for GPT-NeoX-20B is 0.95, against 0.81 for the strongest zero-shot baseline.Paper · 2023-01-24K-Planes: Explicit Radiance Fields in Space, Time, and AppearanceRepresents a d-dimensional radiance field with d-choose-2 learned 2D feature planes, multiplies the bilinear samples across planes, and decodes them to colour and density with a linear learned-basis decoder or a small MLP. Author-reported: close to TensoRF-class methods on static NeRF and LLFF scenes, 33 to 52 min of training on a single GPU for static and monocular dynamic benchmarks.Paper · 2023-01-23HexPlane: A Fast Representation for Dynamic ScenesHexPlane stores a dynamic 3D scene in six learned feature planes: three spatial (XY, XZ, YZ) and three spatio-temporal (ZT, YT, XT). Six bilinear lookups per point are fused by products and concatenation, and a tiny MLP decodes colour inside a NeRF-style renderer. Author-reported: PSNR 29.47 on flame salmon (DyNeRF 29.58) with about 112x less training time.Paper · 2023-01-20A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback AnalysisReview, not a new method: surveys NLP techniques for analysing student feedback text (feature extraction and selection, topic modelling, summarisation, sentiment and entity annotation, knowledge graphs) and the challenges of sarcasm, ambiguity, emoticons, aspect-level sentiment and class imbalance. No experiments; reported accuracies are the cited studies' own.Paper · 2023-01-17GLIGEN: Open-Set Grounded Text-to-Image GenerationGLIGEN keeps a pretrained latent text-to-image diffusion model frozen and adds gated self-attention layers that take boxes (and keypoints, reference images or spatial maps) as extra tokens; the gate starts at zero. Trained on COCO2014, fine-tuned FID is 5.6-6.4 with boxes, and zero-shot LVIS GLIP-score is 6.4 vs 2.0 for supervised LAMA (author-reported).Paper · 2023-01-10Mastering Diverse Domains through World ModelsDreamerV3 learns a recurrent world model (RSSM) from replayed experience and trains an actor and a critic on imagined 15-step rollouts. Symlog inputs, twohot critic outputs, percentile return normalization and free bits keep one fixed configuration across 150+ tasks. Author-reported: Minecraft Diamond return 9.1 at 100M steps, the first agent to collect diamonds from scratch.Paper · 2023-01-10Orbit: A Unified Simulation Framework for Interactive Robot Learning EnvironmentsOrbit is a modular, open-source framework on NVIDIA Isaac Sim for building robot learning environments with GPU-parallel rigid and deformable physics, rendering and range sensors, actuator models and motion generators. Author-reported: about 10x rigid-body throughput over CPU-vectorized frameworks and about 3x cloth throughput over DEDO, plus a sim-to-real locomotion transfer on ANYmal.Paper · 2023-01-09Advances in Medical Image Analysis with Vision Transformers: A Comprehensive ReviewA review, not a method: surveys Transformer and vision-Transformer models for medical imaging across classification, segmentation, reconstruction, synthesis, detection, registration and report generation. Gives per-task taxonomies, strengths and weaknesses, and benchmark tables. Headline: MERIT reaches 84.90 Dice on Synapse (author-compiled table).Paper · 2023-01-05WIRE: Wavelet Implicit Neural RepresentationsWIRE is an implicit neural representation whose MLP uses a complex Gabor wavelet, exp(j w0 x) exp(-|s0 x|^2), as its activation. Its window and frequency terms give sharp, noise-robust fits for images, occupancy volumes, denoising, super-resolution, CT and NeRF. Author-reported: 43.2 dB on a Kodak image; +0.4 dB over SIREN on NeRF drums with 100 images.Paper · 2023-01-02Muse: Text-To-Image Generation via Masked Generative TransformersMuse generates images by predicting randomly masked discrete VQGAN tokens with a transformer, conditioned on frozen T5-XXL text embeddings. A base model and a super-resolution model decode in parallel over a few steps, with classifier-free guidance. Author-reported: FID 6.06 on CC3M (base plus super-res), and zero-shot COCO FID 7.88 with CLIP 0.32 for the 3B model.Paper · 2023-01-02SparseGPT: Massive Language Models Can Be Accurately Pruned in One-ShotSparseGPT prunes every linear layer of a GPT-family model in one pass, without retraining. It picks a mask block by block and corrects the surviving weights with an approximate Hessian-based solver. OPT-175B at 50% unstructured sparsity: WikiText2 perplexity 8.21 vs 8.35 dense (author-reported).Paper · 2022-12-30ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology ReportsExploratory case study: ChatGPT (December 2022 interface) rewrote three fictitious radiology reports in plain language, 15 times each, from one prompt. 15 radiologists rated correctness, completeness and potential harm on 5-point Likert scales. Most judged the rewrites correct, complete and not harmful, but errors, invented findings and missed results were reported. Author-reported; no code.Paper · 2022-12-26Large Language Models Encode Clinical KnowledgeMultiMedQA: six existing medical QA datasets plus HealthSearchQA (3,375 consumer questions), with a human rating framework. Flan-PaLM 540B with few-shot, chain-of-thought and self-consistency prompts reaches 67.6% on MedQA. Med-PaLM adds a 100-token soft prompt trained on 40 clinician exemplars. Author-reported: 92.9% of its answers match scientific consensus, vs 61.9% for Flan-PaLM.Paper · 2022-12-24GraphCast: Learning skillful medium-range global weather forecastingGraphCast forecasts global weather 10 days ahead at 0.25 degrees, six hours per step. A graph neural network encodes the grid onto a multi-scale icosahedral mesh, runs 16 message-passing layers and decodes residual updates, trained with autoregressive MSE on ERA5. Author-reported: it beats HRES-fc0 on 90.3% of 1380 targets.Paper · 2022-12-23The choice of scaling technique matters for classification performanceEmpirical study, not a new method: five feature scalers (standard, min-max, max-abs, robust, quantile) on 82 binary KEEL datasets before 20 classifiers, nine of them ensembles. The scaler choice changes F1 and G-Mean significantly for most models, and a bad choice can be worse than no scaling. Quantile transform is best overall; decision trees and their ensembles barely react. Author-reported.Paper · 2022-12-20Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step QuestionsIRCoT alternates one chain-of-thought sentence with a BM25 retrieval step, using the latest sentence as the next query, until the chain states an answer. Author-reported with GPT-3 on HotpotQA, 2WikiMultihopQA, MuSiQue and IIRC: answer F1 rises over one-step retrieval, e.g. 60.7 vs 53.6 on HotpotQA; Flan-T5 models from 0.7B up also gain.Paper · 2022-12-20Self-Instruct: Aligning Language Models with Self-Generated InstructionsSelf-Instruct bootstraps instruction data from a vanilla GPT-3: 175 hand-written seed tasks prompt the model to write new instructions, classify them, and generate inputs and outputs, filtered by ROUGE-L and heuristics. Fine-tuning GPT-3 on the result raises SuperNI unseen-task ROUGE-L from 6.8 to 39.9, near InstructGPT-001 (40.8). Author-reported.Paper · 2022-12-19Emergent Analogical Reasoning in Large Language ModelsGPT-3 (text-davinci-003) was tested zero-shot on text analogy tasks: a Raven-style Digit Matrices set, letter-string analogies, four-term verbal analogies and story analogies, against undergraduates. Answers come from prompting and log-probability scoring. GPT-3 matched or beat average human accuracy on most tasks, and was weaker on story analogies. Author-reported.Paper · 2022-12-19MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video GenerationTwo coupled U-Nets denoise a short video and its audio jointly. A random-shift cross-modal attention links the two streams, and audio-to-video and video-to-audio conditioning work zero-shot. Author-reported: FVD 117.20 on Landscape and 75.71 on AIST++ with complete DDPM sampling; 84.9% of Landscape clips judged real in a Turing test.Paper · 2022-12-19Scalable Diffusion Models with TransformersReplaces the U-Net in latent diffusion with a ViT-style transformer (DiT) that runs on VAE latent patches and is conditioned by adaLN-Zero. Sample quality (FID) improves as forward-pass Gflops grow, through depth, width or more tokens. DiT-XL/2 reaches FID 2.27 on class-conditional ImageNet 256x256 with classifier-free guidance (author-reported).Paper · 2022-12-15Constitutional AI: Harmlessness from AI FeedbackTrains a helpful, harmless assistant with no human harmlessness labels. Supervised stage: the model critiques and revises its answers to red-team prompts under written principles, then is finetuned on the revisions. RL stage: the model picks the less harmful of two answers to train a preference model for RL (RLAIF). Crowdworkers rate it more harmless than RLHF (author-reported; plots only).Paper · 2022-12-15Rethinking Vision Transformers for MobileNet Size and SpeedEfficientFormerV2 is a family of hybrid vision transformers (S0 to L) built on EfficientFormer, with depthwise convs in the FFNs, self-attention only in the last two stages, and a joint size-speed search. Author-reported: S0 reaches 75.7% top-1 on ImageNet-1K at 3.5M parameters and 0.9 ms on iPhone 12, 3.5 points above a retrained MobileNetV2.Paper · 2022-12-14RTMDet: An Empirical Study of Designing Real-Time Object DetectorsRTMDet is a family of real-time one-stage detectors (tiny to extra-large). It uses CSPNeXt blocks with 5x5 depthwise convolutions, fewer but wider blocks, and a dynamic label assignment with IoU soft labels, trained with cached Mosaic/MixUp. Author-reported: 52.8 AP on COCO val2017 for RTMDet-x at 3.10 ms (FP16, NVIDIA 3090, TensorRT), with instance and rotated variants.Paper · 2022-12-14Reproducible scaling laws for contrastive language-image learningTrains OpenCLIP models (ViT-B/32 to ViT-g/14) on LAION-80M, 400M and 2B for 3B to 34B samples seen, and fits power laws to zero-shot, retrieval, linear-probe and fine-tune error. Best reported: ViT-H/14 on LAION-2B, 78.0% zero-shot ImageNet top-1 (author-reported). Fits extrapolate to 81.9% for ViT-G/14 at 68B samples (not measured). LAION and OpenAI WIT-400M models scale differently by task.Paper · 2022-12-13CREPE: Can Vision-Language Foundation Models Reason Compositionally?CREPE is a retrieval benchmark for compositionality in vision-language models. Systematicity compares seen and unseen compounds for three training sets; productivity uses captions of 4 to 12 atoms with hard negatives. Author-reported: Recall@1 falls on unseen compounds, by up to 12.6 points on atom-foil negatives for LAION-400M models, and falls as caption complexity rises.Paper · 2022-12-10MAGVIT: Masked Generative Video TransformerMAGVIT encodes a video into 4x16x16 discrete tokens with a 3D VQ autoencoder, then trains one BERT-style masked transformer on ten video tasks. Generation takes 12 parallel steps. Author-reported: best FVD on UCF-101 class-conditional (76), BAIR frame prediction (62) and Kinetics-600 (9.9), with about 60x faster inference than autoregressive models.Paper · 2022-12-10ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D UnderstandingULIP pre-trains a 3D point-cloud encoder to share an embedding space with a frozen image-text model (SLIP). It uses ShapeNet55 triplets of rendered images, category captions and point clouds, with contrastive losses on the 3D-image and 3D-text pairs. Author-reported: PointMLP gains about 3% on ScanObjectNN classification; PointBERT+ULIP reaches 60.4% zero-shot top-1 on ModelNet40 (all classes).Paper · 2022-12-09SmartBrush: Text and Shape Guided Object Inpainting with Diffusion ModelSmartBrush inpaints an object into a masked region from a text label. The mask can be exact or coarse, such as a box. Training blurs instance masks to set precision and adds a mask-prediction output to keep the background. Author-reported: lower local FID than Blended Diffusion, GLIDE, Stable Diffusion and Stable Inpainting on OpenImages and MSCOCO, with CLIP scores near theirs.Paper · 2022-12-08Demystifying Prompts in Language Models via Perplexity EstimationLower-perplexity zero-shot prompts tend to score better with OPT and Bloom, with negative correlations on most tasks (not IMDB). SPELL paraphrases seed prompts with GPT-3 and backtranslation, then keeps the 3 lowest-perplexity prompts. Author-reported gains over manual prompts: 1.8 accuracy points (OPT) and 2.3 (Bloom).Paper · 2022-12-08Executing your Commands via Motion Diffusion in Latent SpaceMLD compresses each human motion clip into a short latent with a transformer VAE, then trains a transformer diffusion model in that latent space, conditioned on text (frozen CLIP) or action labels, with classifier-free guidance (s=7.5). Author-reported: FID 0.473 on HumanML3D (MDM 0.544, MotionDiffuse 0.630) and 0.404 on KIT; about 22x less DDIM-50 inference time than MDM.Paper · 2022-12-08LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language ModelsLLM-Planner prompts GPT-3 with the 9 most similar examples from 100 ALFRED training pairs to write high-level subgoal plans, and re-prompts with the objects the agent has seen when it gets stuck. Author-reported ALFRED unseen-test success: 13.41% with goal instructions only and 16.42% with step-by-step instructions, against 20.27% for HLSM trained on all data.Paper · 2022-12-08Multi-Concept Customization of Text-to-Image DiffusionCustom Diffusion teaches Stable Diffusion a new concept from a few images by fine-tuning only the cross-attention key/value projections plus a new modifier-token embedding, regularized with ~200 retrieved real LAION images. Two concepts are combined by joint training or a closed-form constrained least-squares merge. ~6 min on 2 A100s, 75MB per concept, on par with DreamBooth.Paper · 2022-12-08SDFusion: Multimodal 3D Shape Completion, Reconstruction, and GenerationDiffusion model over a 3D VQ-VAE latent of truncated SDF grids. Partial shapes, images (CLIP) and text (BERT) enter by cross-attention with classifier-free guidance and per-input weights; a 2D diffusion model textures the result. Author-reported: lower UHD and higher TMD than MPC and AutoSDF on ShapeNet and BuildingNet completion, best Pix3D F-score (0.432).Paper · 2022-12-08SINE: SINgle Image Editing with Text-to-Image Diffusion ModelsFine-tunes Stable Diffusion on one image with random patch crops and a positional code, then edits by mixing the fine-tuned and pretrained noise predictions for early steps (t > K=400, weight v=0.7). Results are qualitative and author-reported; the text gives no numeric headline.Paper · 2022-12-08The Role of AI in Drug Discovery: Challenges, Opportunities, and StrategiesNarrative review of AI in drug discovery: efficacy and toxicity prediction, de novo design, case studies, researcher-pharma collaboration, data scarcity, bias, data augmentation and explainability. The text was drafted with ChatGPT and rewritten by the human authors as a test. No method or experiments; the only numbers are author-reported draft-similarity and reference-accuracy figures.Paper · 2022-12-07Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion ModelsBuilds a retrieval detector that matches generated images to training images to measure diffusion copying. DINO ViT-B/16 with token-wise split-product similarity is picked from 10 extractors. Small Celeb-A and Flowers models copy often, an ImageNet latent model shows none found, and Stable Diffusion v1.4 has matches above 0.5 for about 1.88% of random samples (author-reported).Paper · 2022-12-07Text Embeddings by Weakly-Supervised Contrastive Pre-trainingContrastive pre-training of a BERT-family bi-encoder with in-batch negatives (batch 32,768, temperature 0.01) on CCPairs, about 270M web and forum text pairs kept by a consistency filter, then optional fine-tuning on NLI, MS-MARCO and NQ. Author-reported: E5-PT-base 42.9 nDCG@10 on BEIR vs BM25 41.7; fine-tuned E5-base 60.4 on MTEB.Paper · 2022-12-06FacT: Factor-Tuning for Lightweight Adaptation on Vision TransformerStacks a ViT's weight matrices into one 3D tensor, factorises the frozen-backbone weight increment with Tensor-Train (FacT-TT) or Tucker (FacT-TK) factors, and trains only those factors plus the head. On VTAB-1K, FacT-TK<=32 averages 75.6% with 0.069M backbone parameters vs NOAH's 75.5% with 0.361M; a 0.008M variant beats full fine-tuning. Author-reported.Paper · 2022-12-06InternVideo: General Video Foundation Models via Generative and Discriminative LearningTrains a masked-video ViT-H and a CLIP-initialised UniFormerV2 video-text encoder separately, then fuses them with learnable cross-model attention on frozen backbones. Author-reported: 91.1% top-1 on Kinetics-400 and 77.2% on Something-Something V2, with state-of-the-art claims on 39 datasets.Paper · 2022-12-05Images Speak in Images: A Generalist Painter for In-Context Visual LearningPainter turns dense vision outputs (depth, segmentation, keypoints, restoration) into RGB images and trains one ViT-L to inpaint the masked output of two stitched pairs from the same task. At inference a solved example pair is the prompt, so no task head is needed. Author-reported: NYUv2 depth RMSE 0.288, which beats the specialist models; ADE20K mIoU 49.9; COCO keypoint AP 72.1.Paper · 2022-12-01Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D GenerationTurns a pretrained 2D diffusion model into a text-to-3D generator without 3D data: a voxel radiance field is rendered from random cameras and the denoiser's score is back-propagated through the renderer. Clean renders are out of distribution for the denoiser, so the score is estimated by adding noise and averaging (NEScore). Stable Diffusion results are qualitative only; no metric is reported.Paper · 2022-11-30Fast Inference from Transformers via Speculative DecodingSpeculative decoding: a small approximation model drafts gamma tokens, the large target model scores all of them in one parallel pass, and a modified rejection rule (speculative sampling) accepts a prefix and resamples one token so outputs are exactly distributed as the target model. 2.3X-3.4X walltime speedup on T5-XXL with a T5-small drafter vs T5X, no retraining.Paper · 2022-11-28Fine-tuning language models to find agreement among humans with diverse preferencesFine-tunes a 70B Chinchilla model to write consensus statements for small groups of people. A reward model predicts each member's agreement with each of 16 sampled statements, and the statement with the highest Utilitarian welfare is shown. Raters preferred it over prompted LLMs and over the best human opinion (author-reported).Paper · 2022-11-28On the Effectiveness of Parameter-Efficient Fine-TuningArgues that random, rule-based and projection-based parameter-efficient fine-tuning all reduce to sparse fine-tuning, which bounds stability and generalization. Proposes SAM: fix the trainable entries by the largest squared gradients from burn-in steps at the pre-trained weights. RoBERTa-base at 0.5% sparsity on 7 GLUE/SuperGLUE tasks: SAM averages 77.12 vs 74.78 for full tuning (author-reported).Paper · 2022-11-26A Physics-informed Diffusion Model for High-fidelity Flow Field ReconstructionTrains a DDPM only on high-fidelity 2D Kolmogorov-flow vorticity, then reconstructs fine fields from coarse or sparse inputs by partial reverse diffusion from a noised guide, optionally guided by the PDE residual gradient. Author-reported: lower equation residual and better energy spectra than a direct-mapping UNet on out-of-distribution inputs, with L2 close to it.Paper · 2022-11-23Vertical Federated Learning: Concepts, Advances and ChallengesSurvey of vertical federated learning, where parties hold different features of the same users. Covers the training protocol, communication and effectiveness methods, privacy attacks and defenses, fairness, applications and open challenges. Proposes VFLow, a meta-objective that treats VFL design as constrained multi-objective optimisation. No new experiments; no official code.Paper · 2022-11-22DETRs with Collaborative Hybrid Assignments TrainingCo-DETR adds training-only auxiliary detection heads (ATSS, Faster R-CNN) with one-to-many label assignment on the DETR encoder output, and feeds their positive boxes back to the shared decoder as extra query groups. Inference is the unchanged DETR. Deformable-DETR R50 (12 ep) goes from 37.1 to 42.9 AP on COCO val; a ViT-L model reaches 66.0 AP on test-dev.Paper · 2022-11-22Plug-and-Play Diffusion Features for Text-Driven Image-to-Image TranslationTraining-free text-guided image-to-image translation with frozen Stable Diffusion: invert the guidance image with DDIM, then while sampling the target prompt from the same noise, overwrite one decoder ResBlock's features and the decoder self-attention queries/keys with those from the guidance pass. Authors report a better structure-vs-text balance than SDEdit, P2P and DiffuseIT.Paper · 2022-11-18Magic3D: High-Resolution Text-to-3D Content CreationMagic3D turns a text prompt into a textured 3D mesh in two stages. A coarse hash-grid neural field is optimized with a 64x64 diffusion prior, then converted to a mesh and refined with a 512x512 latent diffusion prior. Author- reported: 61.7% of user-study raters prefer it to DreamFusion, and a model takes about 40 minutes on 8 A100 GPUs.Paper · 2022-11-18PAL: Program-aided Language ModelsFew-shot prompting makes a code LLM (Codex code-davinci-002) write Python steps with comments instead of final answers; a Python interpreter runs the program to get the answer. No training. Author-reported: 72.0% on GSM8K vs 65.6% for chain-of-thought with the same model; best results on 13 math, symbolic and algorithmic tasks; 80.4% with majority voting over 40 samples.Paper · 2022-11-18SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language ModelsSmoothQuant enables 8-bit weight and activation (W8A8) post-training quantization of LLMs without retraining. Per-channel factors s_j = max|X_j|^α / max|W_j|^(1-α), folded offline into LayerNorm and weights, move outlier difficulty from activations to weights. Author-reported: near-FP16 accuracy on OPT-175B, BLOOM-176B, GLM-130B and MT-NLG 530B; up to 1.56x speedup, 2x memory reduction.Paper · 2022-11-17UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormerAdds UniFormer-style local and global relation blocks to a pretrained image ViT (CLIP by default), so video training starts from image weights without a separate video pretraining stage. Author-reported: 90.0% top-1 on Kinetics-400 (ViT-L/14@336, 64x3x2 views) after Kinetics-710 post-pretraining, and state-of-the-art claims on eight video benchmarks.Paper · 2022-11-16Galactica: A Large Language Model for ScienceTrains decoder-only models from 125M to 120B parameters on a curated 106B-token scientific corpus, with citation, working-memory and molecule/protein tokens and prompts in pre-training. Author-reported: 68.2% on LaTeX equation recall vs GPT-3's 49.0%; 20.4% on MATH vs PaLM 540B's 8.8%; 77.6% on PubMedQA.Paper · 2022-11-15Large Language Models Struggle to Learn Long-Tail KnowledgeRelates closed-book QA accuracy to how many pre-training documents mention both a question's entity and its answer (found by entity linking). Accuracy rises strongly with that count; retraining without those documents shows the effect is causal. Scale helps only log-linearly, and retrieval augmentation reduces the dependence.Paper · 2022-11-14Robot Operating System 2: Design, Architecture, and Uses In The WildReview of ROS 2, the robot middleware built on DDS: design principles, communication patterns, layered architecture, QoS, security and quality process. A small benchmark finds intra-process messaging fastest and default DDS weak above about 1 MB. Five deployments are reported, mostly from interviews and not independently measured.Paper · 2022-11-10InternImage: Exploring Large-Scale Vision Foundation Models with Deformable ConvolutionsInternImage is a CNN backbone built on DCNv3, a 3x3 deformable convolution with shared projection weights, multi-group sampling and softmax-normalised modulation. Scaled to about 1B parameters and pre-trained on large data, it reaches 89.6 top-1 on ImageNet-1K and 65.4 box AP on COCO test-dev (author-reported).Paper · 2022-11-10Unifying Flow, Stereo and Depth EstimationTreats optical flow, rectified stereo and posed-image depth as one dense matching problem. Six cross-attention Transformer blocks build the features, then parameter-free softmax matching gives the output, so one set of weights serves all three. Author-reported: one 1/4-resolution refinement beats RAFT with 31 iterations on Sintel clean and final training EPE.Paper · 2022-11-09BLOOM: A 176B-Parameter Open-Access Multilingual Language ModelBLOOM is a 176B-parameter open-access decoder-only multilingual LLM trained on the ROOTS corpus (46 natural and 13 programming languages). It uses ALiBi positions and an embedding LayerNorm, and was trained in bfloat16 on 384 A100 GPUs. Author-reported: competitive benchmark results, stronger after multitask finetuning (BLOOMZ).Paper · 2022-11-09Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion ModelsSafe latent diffusion (SLD) steers Stable Diffusion away from a text-defined unsafe concept during sampling by adding a safety term to classifier-free guidance. No retraining or external classifier. On the new I2P prompt benchmark, the strongest setting cuts the inappropriate-image probability from 0.39 to 0.09 (author-reported).Paper · 2022-11-05A review of TinyMLA review of TinyML, machine learning on low-power microcontroller-based embedded devices. It describes a generic workflow from data collection to porting a trained model onto a microcontroller, surveys use cases in healthcare, surveillance, AR/VR and industrial maintenance, and lists challenges: low power, limited memory, hardware heterogeneity and missing benchmarks. It reports no experiments.Paper · 2022-11-02eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert DenoisersTrains a text-to-image diffusion cascade as an ensemble of denoisers, each specialised for a band of noise levels and branched from one shared model. Conditions on T5, CLIP text and CLIP image embeddings and adds training-free paint-with-words control. Best zero-shot COCO FID-30K is 6.95 in Table 1 (author-reported).Paper · 2022-11-01Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallReverse-engineers how GPT-2 small completes indirect object identification (after "John gave a drink to", it predicts the name that is not repeated). Path patching and mean ablation find a 26-head circuit in seven classes, tested for faithfulness, completeness and minimality. The circuit keeps 87% of the full model's logit difference.Paper · 2022-11-01MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic ModelApplies a diffusion probabilistic model to medical segmentation: a noisy mask is denoised step by step, conditioned on the image. Adds dynamic conditional encoding (the noisy mask gates image features) and FF-Parser (a learned Fourier-domain filter on skip features). Author-reported Dice, MedSegDiff++: optic cup 87.5, brain tumour 90.5, thyroid nodule 86.6.Paper · 2022-10-31GPTQ: Accurate Post-Training Quantization for Generative Pre-trained TransformersGPTQ quantizes GPT-style model weights layer by layer to 3 or 4 bits, calibrated on 128 text segments. It rounds weights one column at a time and corrects the remaining weights using a Cholesky factor of the inverse Hessian of the layer inputs. Author-reported: small perplexity loss on OPT-175B and BLOOM-176B at 3-4 bits, and about 4 GPU hours per 175B model on one A100.Paper · 2022-10-29MinUn: Accurate ML Inference on MicrocontrollersMinUn is a compiler that generates C++ inference code for ARM microcontrollers. It picks a low or high number precision for each tensor using a search guided by value error and tensor size, then packs tensors into one scratch buffer with an exact-cover solver. Author-reported: 3.81x lower peak RAM than the float SqueezeNet at 54.81% vs 55.02% accuracy.Paper · 2022-10-24Monocular Dynamic View Synthesis: A Reality CheckShows that monocular dynamic view synthesis benchmarks leak multi-view signal through camera teleportation. Proposes effective multi-view factors (EMFs) to measure it, co-visibility-masked image metrics, a keypoint correspondence metric (PCK-T) and an iPhone dataset with non-teleporting, complex-motion captures. Without teleportation, existing methods lose 1-2 dB masked PSNR (author-reported).Paper · 2022-10-24NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance FieldsRuns dense monocular SLAM (DROID-SLAM) for poses and per-pixel depth with marginal covariances, and fits an Instant-NGP radiance field in parallel. The depth loss is weighted by those covariances. Author-reported on Replica: up to 179% higher PSNR and 86% lower depth L1 than Nice-SLAM without depth, at about 10 fps on one 11 GB GPU.Paper · 2022-10-24Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMsProbes GPT-3 on SocialIQa (social commonsense QA) and ToMi (false-belief stories) by k-shot likelihood scoring. Best SocialIQa accuracy is about 55%; ToMi mental-state questions reach about 60%, versus 90-100% on facts. Newer chat models do better on SocialIQa (GPT-4: 79%) but barely improve on mental states. Author-reported.Paper · 2022-10-20Scaling Instruction-Finetuned Language ModelsInstruction finetuning (Flan) on 1,836 tasks, with and without exemplars and with nine chain-of-thought datasets, applied to PaLM, U-PaLM and T5 models up to 540B. Scaling tasks and model size improves held-out scores; Flan-PaLM 540B reaches 75.2% on five-shot MMLU with CoT and self-consistency (author-reported).Paper · 2022-10-17Deep Bidirectional Language-Knowledge Graph PretrainingDRAGON pretrains one encoder jointly on a text segment and a local knowledge-graph subgraph, fusing the two bidirectionally (GreaseLM layers), with masked language modeling plus DistMult link prediction. Author-reported: +7.1 points on OBQA over a continued-pretrained RoBERTa-Large baseline, and 47.5 on MedQA (new best).Paper · 2022-10-17Token Merging: Your ViT But FasterToken Merging (ToMe) merges the most similar tokens in every ViT block, between attention and MLP, via bipartite soft matching on attention keys, size-weighted averaging and proportional attention. Off the shelf, ViT-L/16 MAE at r=8 runs about 2x faster on ImageNet-1k (183 vs 93 im/s) for 1.74 points less top-1 (author-reported). Fine-tuning with it can halve video training time.Paper · 2022-10-16LAION-5B: An open large-scale dataset for training next generation image-text modelsLAION-5B releases 5.85 billion image-text pairs from Common Crawl alt-text, kept by OpenAI CLIP ViT-B/32 similarity (0.28 English, 0.26 other languages) and tagged for NSFW and watermarks. The authors reproduce CLIP on its LAION subsets and show that more data helps zero-shot transfer at equal compute. Results are author-reported.Paper · 2022-10-14Revisiting Heterophily For Graph Neural NetworksAdaptive Channel Mixing (ACM) gives each node a learned mix of three channels inside a GNN layer: low-pass aggregation, high-pass neighbour differences and the node's own features. It also proposes homophily metrics based on post-aggregation similarity. Author-reported: ACM-GCN reaches 94.75% on Cornell and 90.66% on PubMed (random 60/20/20 splits).Paper · 2022-10-12MotionBERT: A Unified Perspective on Learning Human Motion RepresentationsPretrains a DSTformer motion encoder to lift masked, noisy 2D skeleton sequences to 3D, using mocap and in-the-wild videos, then finetunes it with a small head for 3D pose, action recognition and mesh recovery. Reports the lowest scratch 3D pose error among the compared methods and state-of-the-art results on the three tasks (author-reported).Paper · 2022-10-09Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPOVSeg fine-tunes CLIP on masked image regions paired with nouns mined from COCO captions, then adds mask prompt tuning, which swaps blank patches for learnable tokens and can keep CLIP frozen. Plugged into a two-stage MaskFormer segmenter, the Swin-B model reaches 29.6 mIoU on ADE20K-150 when trained on COCO (author-reported), 8.5 above the prior zero-shot state of the art.Paper · 2022-10-07Automatic Chain of Thought Prompting in Large Language ModelsAuto-CoT writes its own few-shot chain-of-thought demonstrations: it k-means clusters the test questions by Sentence-BERT embedding and keeps one zero-shot chain per cluster, nearest the centre and passing length and answer filters. Author-reported on GPT-3 text-davinci-002: at or above Manual-CoT on all ten reasoning datasets (MultiArith 92.0 vs 91.7).Paper · 2022-10-06Binding Language Models in Symbolic LanguagesBinder is a training-free neural-symbolic method. Codex writes a SQL or Python program for a question, with calls back to the same model where the language falls short. The program runs, and majority voting over sampled programs gives the answer. With 14 annotated exemplars it reports 64.6 execution accuracy on WikiTQ test and 85.1 on TabFact small test (author-reported).Paper · 2022-10-06Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question AnsweringRAG-end2end trains RAG's DPR question and passage encoders and its passage index jointly with the BART generator, re-encoding and re-indexing passages in background processes. An auxiliary sentence-reconstruction signal adds domain knowledge. Author-reported exact match with the QA+R setting rises from 3.66 to 8.32 on COVID-19, 8.62 to 14.08 on news and 14.21 to 25.95 on conversations.Paper · 2022-10-06MaPLe: Multi-modal Prompt LearningAdapts a frozen CLIP by learning prompt tokens in both the text and image encoders, linked by a learned linear coupling from language prompts to vision prompts, across the first 9 blocks of each. Trained on 16 shots per class. Author-reported: 78.55% harmonic mean over base and novel classes on 11 datasets, against 75.83% for Co-CoOp.Paper · 2022-10-06Mask3D: Mask Transformer for 3D Semantic Instance SegmentationMask3D predicts 3D instance masks directly with a Transformer decoder. 100 non-parametric instance queries attend to sparse U-Net point features, each within its own predicted mask, and are trained by Hungarian matching with BCE and Dice mask losses. Author-reported 56.6 mAP on ScanNet test, +6.2 over the best listed baseline.Paper · 2022-10-06ReAct: Synergizing Reasoning and Acting in Language ModelsReAct prompts a frozen LLM to interleave short free-text thoughts with actions such as Wikipedia search, lookup and finish, or environment steps. With few-shot PaLM-540B, its best HotpotQA and FEVER scores come from combining it with CoT self-consistency. Reports 71% ALFWorld success (best of six prompt sets) and 40.0% WebShop success. Author-reported.Paper · 2022-10-06VIMA: General Robot Manipulation with Multimodal PromptsVIMA writes robot tasks as prompts that interleave text and object images. A T5 encoder reads the prompt, and a causal transformer decodes discrete pose actions while cross-attending to it. Trained by behaviour cloning on simulated expert data. Author-reported: 81.5% success on the easiest zero-shot level at 200M parameters, versus 58.0% for the best baseline.Paper · 2022-10-05Decomposed Prompting: A Modular Approach for Solving Complex TasksDecomposed Prompting has a decomposer LLM write a program of sub-questions, each sent to its own few-shot prompt, a symbolic function, or a recursive call of itself. Author-reported gains over chain-of-thought on letter concatenation, list reversal, CommaQA and open-domain multi-hop QA; answer extraction lifts GSM8K 36 to 50.6 and MultiArith 78 to 95.Paper · 2022-10-05GLM-130B: An Open Bilingual Pre-trained ModelGLM-130B is a 130B-parameter bilingual (English, Chinese) bidirectional GLM trained by blank infilling on 400B tokens. It uses DeepNorm Post-LN, RoPE, GeGLU and embedding gradient shrink to train stably in FP16. Weight-only INT4 quantization runs it on 4x RTX 3090 with little loss. Author-reported: LAMBADA 80.2 zero-shot, MMLU 5-shot 44.8, above GPT-3 175B on BIG-bench-lite zero-shot.Paper · 2022-10-03Complexity-Based Prompting for Multi-Step ReasoningComplexity-based prompting picks few-shot examples whose reasoning chains have the most steps, and for self-consistency votes only over the K most complex of N sampled chains. Author-reported on GPT-3 and Codex across math word-problem sets, gains over handcrafted chain-of-thought prompts average +5.3 and +6.2 points.Paper · 2022-09-29DiGress: Discrete Denoising diffusion for graph generationDiGress generates graphs by discrete denoising diffusion: node and edge classes are corrupted by Markov category-switching noise, and a graph transformer predicts the clean classes. Adds marginal noise, cycle and spectral features, and property guidance. Author-reported: 74% and 75% valid-unique-novel graphs on SBM and planar benchmarks; on par with autoregressive GraphINVENT on MOSES.Paper · 2022-09-29Human Motion Diffusion ModelMDM is a transformer-encoder diffusion model for human motion. It predicts the clean motion at each step instead of the noise, which lets position, velocity and foot-contact losses be added. It is trained with classifier-free guidance and conditioned on CLIP text, action labels or nothing. Author-reported: HumanML3D FID 0.544 (T2M 1.067), KIT FID 0.497, about 3 days on one RTX 2080 Ti.Paper · 2022-09-28Attention Spiking Neural NetworksMulti-dimensional Attention (MA) adds temporal, channel and spatial attention to the membrane potential of conv-based spiking neurons, with negligible extra parameters. On DVS128 Gesture/Gait the three-dimensional TCSA variant gains 5.9/4.7 accuracy points and cuts spike counts by 84.9%/81.6%. On ImageNet-1K, CSA-Res-SNN-104 reaches 77.08% top-1 with 4 time steps (author-reported).Paper · 2022-09-28Improving alignment of dialogue agents via targeted human judgementsSparrow, an information-seeking dialogue agent trained with RLHF on per-rule violation labels and per-turn preferences, plus Google Search evidence and reranking. Preferred over prompted Chinchilla and more resilient to adversarial probing (author-reported; 8% targeted-rule violations).Paper · 2022-09-26News Summarization and Evaluation in the Era of GPT-3Prompts GPT-3 (text-davinci-002) with a sentence count instead of training on news data, and compares it with fine-tuned BRIO, PEGASUS and T0. Humans prefer the GPT-3 summaries (CNN: 58% majority-best vs 36% for BRIO), but automatic metrics rank them lower, so they cannot reliably judge prompted systems. On keyword-focused summaries, GPT-3 beats CTRLSum in about 70% of pairs.Paper · 2022-09-23Promptagator: Few-shot Dense Retrieval From 8 ExamplesPromptagator prompts a 137B FLAN LLM with 2 to 8 labelled query-document examples to write synthetic queries for each target corpus. A dual encoder trained on them, with round-trip filtering, then retrieves; a 110M reranker adds more. Author-reported: 47.8 nDCG@10 average over 11 BEIR sets, 52.8 with reranking.Paper · 2022-09-22Efficient Few-Shot Learning Without PromptsSetFit fine-tunes a pretrained Sentence Transformer on contrastive positive and negative sentence pairs made from a few labelled examples, then fits a logistic-regression head on the resulting embeddings. No prompts or verbalizers. Author-reported: 62.3 average accuracy with 8 examples per class on five datasets (AGNews excluded), against 63.4 for T-Few 3B, with a 110M-parameter body.Paper · 2022-09-20Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question AnsweringIntroduces ScienceQA, 21,208 multimodal science multiple-choice questions with lectures and explanations, and trains or prompts models to write the answer followed by a lecture and explanation (chain of thought). Author-reported: fine-tuned UnifiedQA improves from 70.12% to 74.11%, and 2-shot GPT-3 from 73.97% to 75.17%.Paper · 2022-09-18RDD2022: A multi-national image dataset for automatic Road Damage DetectionDataset article for RDD2022: 47,420 road photos from six countries, with 55,007 bounding-box labels for four damage types (longitudinal and transverse cracks, alligator cracks, potholes) in the training split. Captured by phones, cameras, street-view and drone; labelled by hand in Pascal VOC XML. No model or benchmark result. Author-reported counts.Paper · 2022-09-15Test-Time Training with Masked AutoencodersFor each test image, a masked autoencoder (ViT-L encoder, ViT-B head) is adapted by 20 SGD steps on masked-patch reconstruction of that image alone, then predicts; weights reset per input. On ImageNet-C level 5 it beats the fixed ViT-probing baseline on all 15 corruptions (author-reported; e.g. JPEG 51.2% to 63.0%, mean not stated).Paper · 2022-09-14Out of One, Many: Using Language Models to Simulate Human SamplesConditions GPT-3 on first-person demographic backstories to make "silicon samples" that stand in for survey respondents. Tested in US politics: judges called GPT-3 and human word lists human at the same rate (61%), vote choice matched ANES 2012-2020 (tetrachoric 0.90-0.94), and cross-item associations matched. Author-reported.Paper · 2022-09-14PaLI: A Jointly-Scaled Multilingual Language-Image ModelPaLI pairs a pretrained mT5 encoder-decoder with a pretrained ViT image encoder and trains it on a multilingual mix of image-text tasks, where every task is image plus text in, text out. Its 17B model (ViT-e, 4B vision) reports 149.1 CIDEr on COCO Karpathy test and 84.3 VQAv2 test-dev, author-reported.Paper · 2022-09-12Perceiver-Actor: A Multi-Task Transformer for Robotic ManipulationPerAct is a language-conditioned behaviour-cloning agent for 6-DoF manipulation. It voxelizes RGB-D input into a 100^3 grid, encodes voxels and a CLIP language goal with a Perceiver Transformer, and predicts the next keyframe action as voxel and binned scores. One multi-task agent covers 18 RLBench tasks; author-reported average gains over a 3D U-Net are 1.33x (10 demos) and 2.83x (100 demos).Paper · 2022-09-05ChemBERTa-2: Towards Chemical Foundation ModelsChemBERTa-2 pretrains a RoBERTa-style transformer on SMILES strings from up to 77M PubChem compounds, comparing masked-token prediction with regression on 200 RDKit descriptors. The property-regression pretraining tends to fine-tune better. Author-reported: on 6 of 8 MoleculeNet tasks it beats the D-MPNN baseline.Paper · 2022-09-02Transformers in Remote Sensing: A SurveySurvey of more than 60 transformer-based methods for remote sensing imagery: very high-resolution optical, hyperspectral and SAR, across classification, detection, change detection, segmentation and other tasks. Collects the benchmark numbers reported by the original papers (for example DOTA mAP up to 80.14, AID 20:80 accuracy up to 97.20) and lists open directions.Paper · 2022-09-01MSGNN: A Spectral Graph Neural Network Based on a Novel Magnetic Signed LaplacianMSGNN is a spectral GNN for signed, directed graphs. It builds a Hermitian magnetic signed Laplacian that carries edge sign and direction, then applies complex Chebyshev filters (K=1, two layers). Author-reported: highest or tied-highest link-prediction accuracy in all 25 cells of five signed/directed tasks (one tie), and generally faster than six baselines.Paper · 2022-09-01Visual Prompting via Image InpaintingAdapts a pretrained masked autoencoder to image-to-image tasks without weight changes: an example pair and a query are tiled into one grid, the bottom-right cell is inpainted and VQGAN-decoded. Pretrained on about 88k arXiv figures. Author-reported 4-split mean mIoU 27.17 on Pascal-5i segmentation, versus 4.28 for ImageNet MAE.Paper · 2022-08-25Understanding Diffusion Models: A Unified PerspectiveTutorial derivation that treats diffusion models as a Markovian hierarchical VAE with fixed Gaussian noising. It shows the ELBO reduces to predicting the clean image, the source noise or the score at each noise level, links this to score-based models and Langevin sampling, and covers classifier and classifier-free guidance. No experiments.Paper · 2022-08-24Semi-Supervised and Unsupervised Deep Visual Learning: A SurveySurvey of visual recognition with unlabeled images. Taxonomies of semi-supervised methods (consistency regularization, self-training, graph regularization, generative models, self-supervision) and unsupervised methods (pretext tasks, instance and cluster discrimination, generative models), linked by three smoothness rationales. Covers segmentation, detection and domain adaptation. No experiments.Paper · 2022-08-23DeepInteraction: 3D Object Detection via Modality InteractionDeepInteraction keeps LiDAR BEV and multi-camera features as separate representations. An encoder lets each side borrow context from the other through 2D-3D projections, and a decoder refines 200 object queries by reading each modality in turn. Author-reported nuScenes test: 70.8 mAP / 73.4 NDS for the ResNet-50 base model, 74.1 / 75.5 with test-time augmentation, 75.6 / 76.3 with ensembling.Paper · 2022-08-23Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons LearnedCrowdworkers chat with four AI assistant types (plain, prompted, rejection-sampled, RLHF; 2.7B to 52B parameters) to elicit harmful replies, then rate each attack. A preference model scores every reply for harm. RLHF models get harder to red team as they grow; the others show no clear trend. Releases 38,961 attacks as data only. Author-reported.Paper · 2022-08-18RFLA: Gaussian Receptive Field based Label Assignment for Tiny Object DetectionLabel assignment for tiny object detection. Each feature point's Gaussian effective receptive field is scored against each ground-truth box with a receptive-field distance (KL divergence or Wasserstein), and a two-stage hierarchical assigner picks positives. Author-reported: Faster R-CNN on AI-TOD rises from 11.1 to 21.1 AP; DetectoRS with RFLA reaches 24.8 AP.Paper · 2022-08-18Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject StudiesTuring Experiments: a language model is prompted zero-shot to act as many different named participants in a human-subject study, and its outcomes are compared with published human results. Four experiments are simulated with OpenAI models; the recent models reproduce the Ultimatum Game, garden-path and Milgram findings, while on a wisdom-of-crowds task they show a hyper-accuracy distortion.Paper · 2022-08-15LLM.int8(): 8-bit Matrix Multiplication for Transformers at ScaleLLM.int8() runs the projection-layer matrix products of transformers in 8-bit integers, without retraining. Vector-wise absmax scales (one per activation row, one per weight column) handle most values; feature dimensions with magnitude of at least 6.0 are multiplied in 16-bit. Author-reported: C4 perplexity matches 16-bit from 125M to 13B parameters; OPT zero-shot accuracy holds up to 175B.Paper · 2022-08-10Patching open-vocabulary models by interpolating weightsPAINT patches a CLIP model for one task. It fine-tunes the visual encoder with a frozen text-derived head, then linearly interpolates weights between the zero-shot and fine-tuned models with one mixing coefficient alpha. On nine tasks where zero-shot CLIP is poor, ViT-L/14 gains 15 to 60 points while ImageNet stays within one point (author-reported). Also covers multi-task and broad transfer.Paper · 2022-08-10Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series ForecastingSTID forecasts multivariate time series with plain MLP layers and no graph. It adds a learned embedding per series (spatial identity) and per time-of-day and day-of-week slot (temporal identity) to the history embedding, then applies residual MLP blocks and a linear head. Author-reported: lowest average MAE on all five benchmarks and the fastest epoch time.Paper · 2022-08-09Simplified State Space Layers for Sequence ModelingS5 replaces S4's bank of single-input, single-output state space models with one multi-input, multi-output diagonal SSM. A parallel scan runs it in the time domain at S4's cost, and HiPPO-N initialisation keeps S4-level accuracy. Author-reported: 87.46% LRA average, 98.58% on Path-X, 94.53% on 8 kHz Speech Commands.Paper · 2022-08-08A review on longitudinal data analysis with random forest in precision medicineReview of CART-based random forest extensions for longitudinal (repeated-measures) data: averaging and subject-level bootstrapping (RF++), historical RF, mixed-effects hybrids (MERF, RE-EM, SMERF, GMERF, BiMM) and multivariate split criteria. Gives model equations, EM-style fitting loops and an R software table; runs no new experiments.Paper · 2022-08-05A Holistic Approach to Undesired Content Detection in the Real WorldBuilds a classifier for eight undesired-content categories: five top-level plus three severe subcategories. It combines precise labeling rules and audits, active learning on production traffic, synthetic data and Wasserstein domain-adversarial training. Author-reported: beats Perspective API on its public test set for sexual, hateful, violent and harassment content.Paper · 2022-08-04Expanding Language-Image Pretrained Models for General Video RecognitionAdapts CLIP-style image-text models to video (X-CLIP) with a cross-frame attention module that exchanges message tokens between frames, a small temporal transformer over frame features, and text prompts generated from each video. Reports 87.1% top-1 on Kinetics-400 with ViT-L/14 at 8 frames, and gains in zero-shot and few-shot settings. Author-reported.Paper · 2022-07-28Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion TransformerInterFuser fuses three RGB views, a focus crop and a LiDAR bird's-eye-view image in one transformer. Decoder queries predict waypoints, a 20x20 object density map and traffic-rule signals; a safety controller caps speed using the map. Author-reported: rank 1 on the CARLA leaderboard, driving score 76.18.Paper · 2022-07-26DETRs with Hybrid MatchingH-DETR trains DETR-style detectors with two query groups: one-to-one Hungarian matching, used alone at inference, and a training-only one-to-many group where each ground truth is repeated K times. Masked self-attention keeps the groups apart. Author-reported: Deformable-DETR R50 (12 epochs) rises from 47.0 to 48.7 AP on COCO val; similar gains on other backbones and DETR variants.Paper · 2022-07-26Representing Random Utility Choice Models with Neural NetworksRUMnet: a neural network that approximates random utility maximization (RUM) discrete choice models. It averages K^2 softmax choice probabilities over sampled unobserved product and customer attributes, then fits by negative log-likelihood. Any RUM model can be approximated arbitrarily closely. On Swissmetro and Expedia, RUMnet has the best accuracy of the benchmarks. Author-reported.Paper · 2022-07-22Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot SegmentationVAT reformulates few-shot segmentation as cost aggregation over the query-support correlation volume. A 4D convolutional Swin Transformer (small overlapping 4D convolutions, then shifted-window attention) runs coarse to fine, followed by an appearance-aware decoder. Author-reported: FSS-1000 1-shot mIoU 90.3 (ResNet-101) vs 86.5 for HSNet.Paper · 2022-07-21TinyViT: Fast Pretraining Distillation for Small Vision TransformersTinyViT pretrains small hierarchical vision transformers on ImageNet-21k by distilling from a large teacher. The teacher's top-K soft labels and the augmentation seeds are stored once on disk, so each student trains without running the teacher. Model sizes come from progressive model contraction. Author-reported: 84.8% ImageNet-1k top-1 with 21M parameters at 224px, and 86.5% at 512px.Paper · 2022-07-18HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image SegmentationHiFormer segments 2D medical images with a ResNet pyramid and a Swin Transformer pyramid, joined by a Double-Level Fusion module that exchanges class tokens between the largest and smallest levels via cross-attention. Author-reported Synapse average Dice: 80.39 for HiFormer-B vs 79.13 for Swin-Unet.Paper · 2022-07-17SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite ImageryPre-trains a masked autoencoder (MAE) on satellite image time series and multi-band images. Each image in a time series gets a year, month and hour encoding, and masking is independent per image. Bands are split into three spectral groups, each with its own patch embedding and a group encoding. Author-reported: fMoW temporal top-1 81.49%, fMoW-Sentinel top-1 61.48%, fMoW RGB fine-tuned 77.84%.Paper · 2022-07-11Language Models (Mostly) Know What They KnowLanguage models are well calibrated on letter-labelled multiple-choice questions. Asking a model whether its own sampled answer is True, P(True), works, and works better when it sees several of its other samples. A value head trained to predict P(IK), the probability that the model knows the answer, reaches AUROC 0.864 on held-out TriviaQA (52B). It generalizes only partly to other tasks.Paper · 2022-07-11No Language Left Behind: Scaling Human-Centered Machine TranslationNLLB-200 is a 54.5B-parameter sparse Mixture-of-Experts translation model for 202 languages. It is trained on human-translated seed data, mined bitext and backtranslation, with expert output masking and a phased curriculum. Author-reported: +7.3 spBLEU over the prior state of the art on FLORES-101 (44% relative); 35.7 average chrF++ on FLORES-200 devtest.Paper · 2022-07-10LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and ActionLM-Nav follows free-form route instructions on a ground robot without fine-tuning or language-labelled robot data. A frozen GPT-3 extracts ordered landmarks, CLIP scores which map nodes match each landmark, and a graph search finds the walk that visits them in order at low travel cost. Author-reported: 0.8 net success over 20 outdoor instructions (6 km) on a Clearpath Jackal.Paper · 2022-07-06FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment SamplingScores video quality from fragments: a grid of raw-resolution mini-patches, sampled at fixed offsets across all frames and spliced into one canvas. A Swin-T backbone with gated relative position biases and a patch-wise regression head is trained end to end. Author-reported: SRCC 0.779 and PLCC 0.814 on LSVQ 1080p, using about 0.5% of PVQ's FLOPs at 1080P.Paper · 2022-07-04WebShop: Towards Scalable Real-World Web Interaction with Grounded Language AgentsWebShop is a simulated Amazon-style shopping site with 1.18M products and 12,087 human instructions. Agents search, pick options and buy; partial-credit reward. Imitation learning (BART search, BERT choice) plus policy-gradient RL reaches 62.4 score and 28.7% success, below expert humans (59.6%).Paper · 2022-06-30Causal Machine Learning: A Survey and Open ProblemsSurvey of causal machine learning. Introduces SCMs, interventions and counterfactuals, then groups methods into five problem areas (supervised learning, generative modeling, explanations, fairness, RL) and reviews causal benchmarks and open problems. Key method families: invariant risk minimization, DeepSCM, counterfactual fairness. No experiments or code of its own.Paper · 2022-06-30PolarFormer: Multi-camera 3D Object Detection with Polar TransformerDetects 3D objects from six surround cameras on a polar bird's-eye-view grid (radius and azimuth). Cross-attention turns image columns into polar rays; a deformable-attention decoder predicts boxes whose heading and velocity are measured relative to azimuth. Author-reported nuScenes val: 39.6 mAP / 45.8 NDS (ResNet-101), 50.0 / 56.2 (VoVNet-V2-99 with extra pretraining).Paper · 2022-06-29Solving Quantitative Reasoning Problems with Language ModelsMinerva continues training PaLM 8B, 62B and 540B on 38.5B tokens of arXiv papers and maths-heavy web pages, then answers without external tools. Final answers come from few-shot chain-of-thought, with majority voting over sampled solutions (maj1@k). At 540B, maj1@k reaches 50.3% on MATH, 78.5% on GSM8k and 75.0% on MMLU-STEM (author-reported).Paper · 2022-06-27ProGen2: Exploring the Boundaries of Protein Language ModelsNext-token transformers for protein sequences, from 151M to 6.4B parameters, trained on UniRef90, BFD, metagenomic and antibody (OAS) sequences. Larger models lower held-out perplexity, but zero-shot fitness ranking peaks at 764M. Generated sequences are often predicted to fold like natural proteins at low sequence identity. Author-reported.Paper · 2022-06-27ST-Adapter: Parameter-Efficient Image-to-Video Transfer LearningST-Adapter adds a small bottleneck module to each block of a frozen image ViT. A depth-wise 3D convolution inside it mixes features across frames, so the image model can classify video. Only the adapters and a linear head train: about 7.2M parameters for ViT-B/16. Author-reported: 82.0% top-1 on Kinetics-400 (full fine-tuning 81.7%) and 66.3% on Something-Something-v2 (66.1%).Paper · 2022-06-21EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision ApplicationsEdgeNeXt is a mobile hybrid CNN-transformer. Its SDTA block splits channels into groups, applies depth-wise convolutions across them, and runs self-attention across channels, so attention cost grows linearly with image size. Author-reported: 79.4% ImageNet-1K top-1 with 5.6M parameters and 1.3G MAdds at 256px, above MobileViT-S's 78.4%.Paper · 2022-06-21Using cognitive psychology to understand GPT-3Tests GPT-3 (Davinci, temperature 0) on classic cognitive-psychology experiments: vignettes, decisions from descriptions, a horizon bandit task, the two-step task and causal reasoning. GPT-3 makes human-like choices on many tasks and shows model-based signatures, but does not explore directedly, is brittle to small prompt changes, and fails the causal task.Paper · 2022-06-20The Fallacy of AI FunctionalityArgues that many deployed AI systems do not work, so policy should first ask whether a system functions. From 283 incidents (2012-2021) in the AIAAIC repository it builds a taxonomy of functionality failures and maps existing legal and organizational levers. A position paper: no experiments, no code.Paper · 2022-06-193D Object Detection for Autonomous Driving: A Comprehensive SurveySurvey of 3D object detection for autonomous driving, organised by sensor (LiDAR, camera, multi-modal), by temporal and label-efficient settings, and by driving-system use. Compiles author-reported KITTI, nuScenes and Waymo results and inference times. Headline: LiDAR and fusion methods lead on accuracy; the best monocular method reaches 16.34% moderate AP3D on KITTI.Paper · 2022-06-18Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series ForecastingSTEP pre-trains a small Transformer (TSFormer) to reconstruct masked 12-step patches of one to two weeks of traffic history, then feeds its patch states and a Gumbel-sampled sparse graph into Graph WaveNet. Author-reported: best in nearly all cells on METR-LA, PEMS-BAY and PEMS04 against 12 baselines.Paper · 2022-06-17Diffusion models as plug-and-play priorsUses an independently trained denoising diffusion model as a prior. A point estimate is optimised by gradient steps on the denoising error plus a differentiable constraint, with the noise level annealed from high to low. Author-reported: 1.28% tour gap on 50-node TSP without beam search, and land-cover transfer comparable to prior work.Paper · 2022-06-17MineDojo: Building Open-Ended Embodied Agents with Internet-Scale KnowledgeTrains a Minecraft agent on free-form task prompts with MineCLIP, a video-text contrastive model pre-trained on 640K YouTube clip-transcript pairs, used as a dense reward for PPO with self-imitation. Author-reported: close to hand-made rewards on 8 Programmatic tasks (paired t-test p=0.3991) and well above CLIP-guided agents on 4 Creative tasks.Paper · 2022-06-17Unified-IO: A unified model for vision, language, and multi-modal tasksUnified-IO turns images, masks, boxes, keypoints, depth maps and text into one vocabulary of discrete tokens, then trains a single T5-style encoder-decoder on 95 datasets with no task-specific heads. The 2.9B-parameter XL model reports 64.3 average on the GRIT test set, against 32.0 for the next entry. All numbers are author-reported.Paper · 2022-06-16Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong BaselineCombines a waypoint branch and a direct control branch in one network for end-to-end driving from a single front camera. The control branch uses attention guided by the waypoint branch, and the two outputs are blended by a turning rule. Trained by imitating a Roach RL expert. Author-reported: driving score 75.137 (ensemble), first on the CARLA leaderboard.Paper · 2022-06-15Contrastive Learning as Goal-Conditioned Reinforcement LearningShows that contrastive learning on action-labelled trajectories is a goal-conditioned RL method: the inner product of state-action and goal representations equals a Q-function up to a constant. The simple variant, contrastive RL (NCE), needs no augmentation or auxiliary losses. Author-reported: it beats HER, goal-conditioned BC and model-based baselines on most tasks.Paper · 2022-06-15Emergent Abilities of Large Language ModelsSurvey that defines an emergent ability as one absent in smaller language models but present in larger ones, so it cannot be extrapolated from small-scale curves. Catalogs few-shot tasks (arithmetic, MMLU, WiC, TruthfulQA, many BIG-Bench tasks) and prompting/finetuning methods (chain of thought, instruction tuning, scratchpad) whose gains appear only above roughly 1e22 to 1e24 training FLOPs.Paper · 2022-06-14Deep Isolation Forest for Anomaly DetectionDeep Isolation Forest passes data through 50 random, untrained neural-network representations and grows 6 isolation trees on each, using random axis-parallel cuts. Scores combine path depth with how far points sit from their cuts. Author-reported: average AUC-PR 0.351 on ten tabular sets, against 0.219 for EIF and 0.144 for iForest. Also tested on graphs and time series.Paper · 2022-06-13Memory-Based Model Editing at ScaleModel editing without touching the base model's weights. SERAC stores edit descriptors in a cache, uses a scope classifier to decide whether an input falls under an edit, and routes such inputs to a small counterfactual model. Author-reported: highest edit success on QA, QA-hard, FC and ConvSent with 10 simultaneous edits (5 on ConvSent), with higher drawdown on FC.Paper · 2022-06-10Measuring the Carbon Intensity of AI in Cloud InstancesEstimates the operational CO2 of GPU cloud jobs as GPU energy times time-specific marginal grid intensity (WattTime data, Microsoft Azure, 16 regions). Measures 11 NLP and vision workloads, then simulates Flexible Start and Pause-and-Resume scheduling. Region choice matters most; Pause-and-Resume cuts emissions by up to 27.7% (DenseNet 121, 24 h extension), author-reported.Paper · 2022-06-09Factuality Enhanced Language Models for Open-Ended Text GenerationMeasures and reduces nonfactual text from open-ended LMs: FactualityPrompts with Wikipedia-based NE and NLI metrics, factual-nucleus sampling (top-p with a per-sentence decay and floor), and continued Wikipedia training with a title prefix and sentence-completion loss. Author-reported: 530B named-entity error on factual prompts falls from 33.3% (top-p) to 14.5% with both changes.Paper · 2022-06-06Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of ExpertsLIMoE is a sparse mixture-of-experts model that takes images and text in one shared transformer, trained with a contrastive loss. Two entropy-based routing losses, plus batch priority routing, keep experts from collapsing when the two modalities are mixed. The largest model, LIMoE-H/14 (5.6B parameters, 675M used per token), reports 84.1% zero-shot ImageNet accuracy, author-reported.Paper · 2022-06-03Metrics reloaded: Recommendations for image analysis validationMetrics Reloaded is a Delphi-consensus decision framework for choosing validation metrics for image-level classification, object detection, semantic and instance segmentation. A problem fingerprint drives a decision tree to metric pools, with pitfalls flagged. No benchmark numbers; author-reported consensus above 75% for all ten components.Paper · 2022-06-02EfficientFormer: Vision Transformers at MobileNet SpeedEfficientFormer is a pure vision transformer built to run as fast as MobileNet on phones. It uses a convolutional stem and pooling blocks in early stages, global self-attention only in the last stage, and picks depth and width by latency-driven slimming. Author-reported: 79.2% ImageNet top-1 at 1.6 ms on iPhone 12 (L1), 83.3% at 7.0 ms (L7).Paper · 2022-05-30Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous VehiclesEdge YOLO is a slimmed YOLOv4-style detector: a CSPDarknet53 backbone cut to 17 layers, an FPN neck and three 13/26/52 heads, run on a Jetson-class edge device while a cloud server retrains it. Author-reported: 26.6 FPS on Jetson Xavier and 0.473 mAP@0.5 on COCO2017 (YOLOv4: 0.657).Paper · 2022-05-30Multi-Game Decision TransformersOne GPT-2-style transformer, trained offline on Atari trajectories from 41 games, predicts return, action and reward tokens from 84x84 frames cut into 14x14 patches. At play time it samples a high target return, then an action. Author-reported: 126% of human-normalised IQM across the 41 games, gains with model size, and fast fine-tuning to 5 held-out games.Paper · 2022-05-30Prompt-aligned Gradient for Prompt TuningLearns CLIP prompt vectors as CoOp does, but removes from each cross-entropy gradient the part that conflicts with the gradient of a KL term toward zero-shot CLIP predictions. Author-reported, ResNet-50, 11-dataset average: 62.61% vs CoOp 59.46% at 1 shot, 74.28% vs 73.49% at 16 shots.Paper · 2022-05-30You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure CorrectionIAT restores low-light and under- or over-exposed sRGB photos with about 90k parameters. A small convolutional local branch predicts per-pixel multiply and add maps, and attention queries predict a 3x3 colour matrix and a gamma for the whole image. Author-reported: PSNR 23.50 and SSIM 0.824 on LOL-V2-real, at about 0.004 s per image.Paper · 2022-05-27BEVFusion: A Simple and Robust LiDAR-Camera Fusion FrameworkBEVFusion gives the camera its own stream: image features are lifted into a bird's-eye-view grid with predicted depth and fused with a LiDAR BEV feature. The camera stream never needs LiDAR points, so the detector still predicts when LiDAR is dropped or cropped. Author-reported: 69.2 mAP and 71.8 NDS on nuScenes test.Paper · 2022-05-27Diffusion-LM Improves Controllable Text GenerationDiffusion-LM is a non-autoregressive language model: it denoises a sequence of Gaussian vectors into word embeddings, trained end to end with a rounding loss. Its continuous latents let gradient steps from attribute classifiers steer the output, including syntax-tree and span controls. Author-reported: semantic-content control success 81.2% vs 69.9% for FUDGE; POS control 90.0% vs 27.0%.Paper · 2022-05-27Surrogate modeling for Bayesian optimization beyond a single Gaussian processEGP-TS keeps an ensemble of Gaussian processes with different kernels for Bayesian optimization, weights them by fit to the data, and picks each query by Thompson sampling from the weighted mixture, using random features. Author-reported: lower simple regret than single-GP Thompson sampling, GP-EI and two bandit ensembles on synthetic and robot-pushing tasks; sublinear Bayesian regret.Paper · 2022-05-27Tranception: protein fitness prediction with autoregressive transformers and inference-time retrievalTranception is an autoregressive protein transformer (up to 700M parameters) with head-grouped convolutional attention and grouped ALiBi positions. At inference it mixes its residue probabilities with a retrieved-alignment profile (weight 0.6), scoring both reading directions. On ProteinGym substitutions it reaches Spearman 0.451 vs EVE's 0.448 (author-reported).Paper · 2022-05-26Are Transformers Effective for Time Series Forecasting?LTSF-Linear: one linear layer maps the last 336 values of each variate to the next T values in one shot. DLinear adds a trend/seasonal split, NLinear subtracts the last value. On nine benchmarks the best linear model has lower MSE than the best Transformer forecaster in all 36 dataset-horizon cells, often by a wide margin. Author-reported.Paper · 2022-05-26Fast Vision Transformers with HiLo AttentionHiLo splits an attention layer's heads into a local group (self-attention in small windows, for fine detail) and a global group (queries attend to window-averaged keys and values, for coarse structure). LITv2 uses it with depthwise-conv FFNs instead of relative position bias. Author-reported: LITv2-S gets 82.0% ImageNet top-1 at 1,471 images/s on an RTX 3090.Paper · 2022-05-25FLEURS: Few-shot Learning Evaluation of Universal Representations of SpeechFLEURS: about 1.4k hours of read speech in 102 languages, n-way parallel through FLoRes-101 sentences, with three native-speaker recordings per sentence and validation by other workers. Baselines: 600M w2v-BERT (speech-only) and mSLAM (speech+text) for ASR and language ID; mSLAM alone for retrieval. Average CER 14.1 (w2v-BERT) vs 14.6 (mSLAM).Paper · 2022-05-24OnePose: One-Shot Object Pose Estimation without CAD ModelsEstimates the 6D pose of an object from a short RGB video scan, without CAD models or per-object training. The scan becomes a sparse SfM map; a graph attention network matches query 2D features directly to its 3D points, and PnP with RANSAC gives the pose. Author-reported: on 150 objects it is on par with HLoc (SuperPoint + SuperGlue) at 58 ms vs 618 ms per frame, and beats PVNet and Objectron.Paper · 2022-05-24The Authenticity Gap in Human EvaluationArgues that averaging Likert ratings can reverse true annotator preferences, and proposes system-level probabilistic assessment (SPA): annotators estimate the probability they prefer one system over another. On GPT-3 story generation, the authors report SPA recovers 5 of 5 expected preferences, against 2 of 5 for the standard protocol.Paper · 2022-05-21Knowledge Distillation from A Stronger TeacherDIST distills from a stronger teacher by matching only the Pearson correlation of softmax outputs: across classes per instance and across instances per class, instead of the exact KL match. Author-reported: ResNet-18 student from a ResNet-34 teacher reaches 72.07% top-1 on ImageNet (KD 70.66%); gains also on COCO and Cityscapes.Paper · 2022-05-21Self-Supervised Speech Representation Learning: A ReviewA review of self-supervised speech representation learning. It groups methods into generative (rebuild masked or future input), contrastive (CPC, wav2vec 2.0) and predictive (HuBERT, WavLM, data2vec, which learn from teacher targets). It also covers datasets, the SUPERB benchmark, analysis of learned features and zero-resource speech. Reports no new method; numbers are author-reported.Paper · 2022-05-20Can Foundation Models Wrangle Your Data?Turns five data cleaning and integration tasks (entity matching, imputation, error detection, data transformation, schema matching) into text prompts for GPT-3 175B, with rows written as attribute: value and 0 to 10 worked examples, no training. Author-reported: matches or beats prior state of the art on several benchmarks, some as ties. Wording and example choice shift F1 by 10 to 15 points.Paper · 2022-05-20Deep transfer learning for image classification: a surveySurvey of deep transfer learning for image classification. Defines it, then sorts methods by source-target similarity and target size: instance, feature, parameter (regularisation, normalisation, ensembles) and best-practice guidance. Its one own experiment shows Big Transfer's fine-tuning settings are far from optimal for less similar targets (Cars 81.8 to 87.6 mean).Paper · 2022-05-19Mitigating Neural Network Overconfidence with Logit NormalizationLogitNorm trains with cross-entropy on logits divided by their L2 norm and a temperature tau, so the logit vector keeps a fixed length during training and cannot grow to inflate confidence. Author-reported on CIFAR-10 with WRN-40-2: the average FPR95 of the softmax score falls from 49.52% to 15.65% over six OOD sets, with ID accuracy close to cross-entropy.Paper · 2022-05-19Selection-Inference: Exploiting Large Language Models for Interpretable Logical ReasoningSelection-Inference (SI) loops a frozen LLM: a selection step picks facts from the context, an inference step derives one new fact from only those facts, and the fact is added back. With a 7B Gopher model, 5-shot, no fine-tuning, the author-reported average over 10 logic tasks is 58.75% (generative), vs 2.94% naive and 41.32% chain-of-thought.Paper · 2022-05-15FreeMatch: Self-adaptive Thresholding for Semi-supervised LearningFreeMatch sets the pseudo-label confidence threshold of semi-supervised image classifiers from the model's own running confidence: a global EMA threshold scaled per class by an EMA of class probabilities. A self-adaptive fairness term keeps predictions diverse. Author-reported: error 8.07% vs FlexMatch 13.85% on CIFAR-10 with 10 labels; 15.56% vs 29.15% on STL-10 with 40 labels.Paper · 2022-05-13A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and OpportunitiesSurvey of recent few-shot learning (FSL) work, organised by the challenge each method addresses: data augmentation, transfer learning, meta-learning and multimodal learning. It adds cross-domain FSL, metric-learning and graph-network summaries, and computer-vision tables for classification, detection and segmentation. The survey runs no experiments; its numbers are copied from cited papers.Paper · 2022-05-11Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context LearningT-Few fine-tunes T0 (11B) with (IA)3, learned vectors that rescale keys, values and FFN activations, plus unlikelihood and length-normalized losses. Author-reported: beats few-shot GPT-3 175B on held-out tasks at about 1,000x lower inference FLOPs, and is the first to beat the RAFT human baseline.Paper · 2022-05-09Panoptic Neural Fields: A Semantic Object-Aware Neural Scene RepresentationPanoptic Neural Fields (PNF) model a driving scene as one small MLP per tracked object, placed in a learned 3D box, plus a background MLP with a semantic head. They are trained by volume rendering against RGB images and predicted 2D segmentations. Object MLPs can start from a ShapeNet car prior learned by FedAvg. Author-reported on KITTI-360 novel views: 74.28 mIoU and 21.91 PSNR.Paper · 2022-05-06EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision TransformersEdgeViT is a family of light ViTs (about 0.6 to 1.9 GFLOPs) built on a local-global-local block: convolutions aggregate neighbours, attention runs only on one sampled token per window, and a transposed convolution spreads it back. Author-reported: EdgeViT-XXS reaches 74.4% ImageNet-1K top-1 at 32.8 ms on a Galaxy S21 CPU, and the models are Pareto-optimal on latency and energy.Paper · 2022-05-05Cross-view Transformers for real-time Map-view Semantic SegmentationCross-view Transformer segments the map view (bird's-eye view) of a car from six camera images. Each camera gets a positional embedding built from its calibration, and cross-attention links learned map cells to image locations, so no explicit depth is estimated. Author-reported on nuScenes vehicle segmentation: IoU 36.0 at 5M parameters and 35 FPS on an RTX 2080 Ti.Paper · 2022-05-02OPT: Open Pre-trained Transformer Language ModelsOPT is a family of decoder-only GPT-3-style Transformers from 125M to 175B parameters, trained on about 180B tokens of deduplicated web, book, Reddit and other text, with a published logbook and training code. Author-reported: OPT-175B is comparable to GPT-3 on zero- and few-shot NLP averages at about one seventh of its carbon footprint.Paper · 2022-04-28Deep Generalized Unfolding Networks for Image RestorationDGUNet unfolds Proximal Gradient Descent into a 7-stage CNN for image restoration. Each stage learns its gradient step, with two residual blocks standing in for an unknown degradation operator and its transpose, plus a U-Net-style proximal mapping. Author-reported: DGUNet+ beats compared methods on deraining, deblurring and denoising test sets; +0.73 dB PSNR over MPRNet on five deraining sets.Paper · 2022-04-28Transformers in Time-series Analysis: A TutorialTutorial on Transformers for time-series analysis. Explains self-attention, multi-head attention, encoder/decoder blocks and sinusoidal positional encoding, surveys time-series adaptations (Informer, LogSparse, SAnD, Traffic Transformer and others) and gives training advice. It runs no experiments of its own; its performance statements are qualitative and come from cited works.Paper · 2022-04-27Attention Mechanism in Neural Networks: Where it Comes and Where it GoesSurvey of attention in neural networks, from late-1980s visual attention models through additive, soft/hard, global/local and scaled dot-product (Transformer) attention to its variants and quadratic-cost fixes. Gives the core formulas. No experiments, no code, no benchmark numbers.Paper · 2022-04-25Trusted Multi-View Classification with Dynamic Evidential FusionEach view's network outputs non-negative evidence, which becomes a Dirichlet with belief and uncertainty masses. Views are fused with a reduced Dempster-Shafer rule, so uncertain views count less. A KL term regularises each view. Author-reported: 61.3% fused accuracy on SUN RGB-D and 72.5% on NYUD2 (ResNet-18), plus gains on six feature-vector datasets.Paper · 2022-04-21Contrastive Test-Time AdaptationAdapts a source-trained classifier to unlabelled target data without source images. Trains a MoCo-style contrastive loss jointly with cross-entropy on pseudo labels refined per batch by soft nearest-neighbour voting over a memory queue, excluding same-pseudo-label negatives. Author-reported: 86.8% per-class average on VisDA-C (SHOT 83.0%), 67.8% on DomainNet-126 over 7 shifts (SHOT 67.1%).Paper · 2022-04-19ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual ModelsBenchmarks task-level transfer of language-augmented visual models on 20 classification and 35 detection datasets, with WordNet, Wiktionary and GPT-3 class knowledge and automatic hyper-parameter search. Proposes starting the linear head from the text encoder. Author-reported: CLIP ViT-B/32 linear probing at 5 shots averages 65.35% over 20 IC datasets, against 58.09% with a random head.Paper · 2022-04-19Image Data Augmentation for Deep Learning: A SurveySurvey of image data augmentation covering manipulation, erasing, mixing, automated policy search, feature-space and GAN methods. Its own runs on PASCAL VOC segmentation, CIFAR-10/100, SVHN and COCO2017 use only the manipulation, erasing and mixing methods. Author-reported gains of 0.4 to 2.8 points; CenterNet detection is slightly lower with augmentation.Paper · 2022-04-18VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language GuidanceMakes images from text, or edits an image with a text instruction, without training a model. A pretrained VQGAN decoder's latent grid is optimised with Adam so that CLIP scores the image closer to the prompt, averaging over many augmented crops per step. Author-reported human ratings (1-5, 12 prompts): mean 4.6 for VQGAN-CLIP vs 3.3 for the best GLIDE baseline.Paper · 2022-04-17Anomaly Detection in Autonomous Driving: A SurveySurvey of anomaly (corner case) detection for autonomous driving across camera, lidar, radar, multimodal and object-level data. It classifies 49 methods by detection approach, corner case level and online capability. Most work is camera-based and scene-level; lidar and radar lag. The survey runs no experiments of its own.Paper · 2022-04-16Visual Attention Methods in Deep Learning: An In-Depth SurveyA survey of attention mechanisms for vision models. It groups about 50 to 70 methods by how they compute attention scores (soft or hard, then channel, spatial, self, arithmetic, multi-modal, logical, statistical, reinforcement, Gaussian and clustering) and lists formulas, strengths, limitations and open problems. It has no experiments of its own; quoted figures come from the original papers.Paper · 2022-04-15mGPT: Few-Shot Learners Go MultilingualTwo GPT-3-style decoder-only models, mGPT-1.3B and mGPT-13B, pretrained with a causal language-modelling objective on Wikipedia and C4 in 61 languages. Zero- and few-shot tasks are scored by label likelihood. Author-reported: mGPT-1.3B is comparable to XGLM-1.7B on classification, while mGPT-13B trails XGLM-7.5B on most few-shot tasks.Paper · 2022-04-14BEHAVE: Dataset and Method for Tracking Human Object InteractionsTracks a person, a rigid object with a known template mesh, and their contact regions per frame from four calibrated RGBD cameras. Networks predict unsigned distances and SMPL correspondences at sampled 3D points, and a joint fit uses them. Author-reported: SMPL vertex error 4.99 cm and object vertex error 21.20 cm, against 13.73 cm and 34.73 cm for PHOSA.Paper · 2022-04-14Masked Siamese Networks for Label-Efficient LearningMasked Siamese Networks (MSN) pre-trains a Vision Transformer without labels. A randomly masked view must match the prototype assignments that an EMA teacher gives the unmasked view, so no pixels are reconstructed. Author-reported: 72.4% top-1 with 5 labels per ImageNet class and 75.7% with 1% of labels (both ViT-B/4).Paper · 2022-04-14YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity LossYOLO-Pose adds a keypoint head to a YOLOv5 detector so that each person's box and 17 COCO keypoints come out of one forward pass, trained end to end with an OKS-based keypoint loss. Author-reported: 90.2 AP50 on COCO val2017 and 90.3 AP50 on test-dev2017 with the largest model, without test-time augmentation.Paper · 2022-04-13Hierarchical Text-Conditional Image Generation with CLIP LatentsunCLIP: a text-to-image model in two stages. A prior maps a caption to a CLIP image embedding (diffusion or autoregressive), and a GLIDE-style diffusion decoder inverts the CLIP image encoder to render it. The embedding gives more diverse samples than GLIDE at similar realism, and the diffusion prior beats the AR prior. Author-reported: zero-shot MS-COCO FID 10.39.Paper · 2022-04-13Single-grasp Deformable Object Discrimination: the Effect of Gripper Morphology, Sensing Modalities, and Action ParametersFour robot hands and grippers squeeze two sets of deformable objects once each. Classifiers (SVM on features, LSTM on raw time series, k-NN) learn the object from the position and effort or tactile readings. The Barrett Hand reaches 94-99% on 20 polyurethane foams with SVM or LSTM. Faster squeezes and changed finger configurations hurt generalisation. Author-reported.Paper · 2022-04-12SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detectionSwinNet uses two Swin-B encoders, for RGB plus depth or thermal input. Per level, spatial and channel re-calibration aligns the streams, and an edge branch from shallow depth features guides the decoder. Author-reported: best S-measure on NLPR (0.941) and NJU2K (0.935) among 19 RGB-D baselines, and on VT5000 (0.912) among 9 RGB-T baselines.Paper · 2022-04-12Training a Helpful and Harmless Assistant with Reinforcement Learning from Human FeedbackTrains chat assistants to be helpful and harmless with preference models fitted to crowdworker comparisons, then RLHF (PPO with a KL penalty) up to 52B parameters. Iterated online retraining with fresh feedback improves crowd-rated helpfulness. Author-reported: 86% PM accuracy on an HHH benchmark; helpfulness and harmlessness pull against each other.Paper · 2022-04-09Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic LoadsTrains proprioception-only PPO walking policies for the bipedal robot Cassie under unloaded and four dynamic loads (tray box, cart, carrying pole, water jug). Training with the load in simulation is enough; a general load policy and bootstrapping from a base walker also work. Carrying-pole pass rate rises from 0.0014 to 0.9578 in simulation, but hardware top speeds fall well below simulation.Paper · 2022-04-07Winoground: Probing Vision and Language Models for Visio-Linguistic CompositionalityWinoground is a 400-example, expert-written benchmark. Each item has two images and two captions with the same words in a different order, and a model must match captions to images. It defines text, image and group scores. Of the ten transformer and two RNN families tested, the best group score is 14.50 percent, against 85.50 percent for humans. Author-reported.Paper · 2022-04-05Multi-View Transformer for 3D Visual GroundingMulti-View Transformer (MVT) grounds a sentence to an object in a 3D point-cloud scene. It places each object at several rotated views of the scene, fuses each view with the sentence in a transformer decoder, then averages over views. Author-reported: 55.1% on Nr3D (+11.2 points over the best non-assisted baseline) and 64.5% on Sr3D (+7.1).Paper · 2022-04-05PaLM: Scaling Language Modeling with PathwaysPaLM is a 540B-parameter dense decoder-only Transformer (SwiGLU, parallel attention+MLP blocks, multi-query attention, RoPE, no biases) trained on 780B tokens across 6144 TPU v4 chips without pipeline parallelism, at 46.2% model FLOPs utilization. Author-reported: few-shot SOTA on 28 of 29 English NLP benchmarks and 58% on GSM8K with chain-of-thought.Paper · 2022-04-05Text2LIVE: Text-Driven Layered Image and Video EditingText2LIVE edits the look of existing objects in one image or video from a text prompt, without a pretrained generator or masks. A small CNN is trained on augmented copies of the single input to output an RGBA edit layer composited over it, guided by CLIP losses. Videos are edited through a neural layered atlas. Author-reported: 73-86% of human votes favour it over baselines.Paper · 2022-04-05iSDF: Real-Time Neural Signed Distance Fields for Robot PerceptioniSDF trains a randomly initialised MLP online on posed depth images to approximate a signed distance field, self-supervised by bounding predictions with the nearest surface point in a sampled batch. Author-reported: under 6 cm SDF error at sequence end on 6 main sequences, about 33 ms per iteration, and lower error than Voxblox and KinectFusion+.Paper · 2022-04-04A Survey on Graph Representation Learning MethodsSurvey of graph embedding methods for static and dynamic graphs, split into traditional methods (factorization, random walks, autoencoders, temporal point processes) and GNN-based methods (recurrent, convolutional, spatio-temporal, dynamic GNNs). It also reviews nine GNN limitations and proposed fixes. It runs no experiments; the authors report covering over 300 papers.Paper · 2022-04-03BinsFormer: Revisiting Adaptive Bins for Monocular Depth EstimationBinsFormer predicts depth as a per-pixel mix of adaptive bin centres. A Transformer decoder generates the bins from learned queries, one extra query classifies the scene, and features are refined at three scales. Author-reported: on NYU Depth v2 with a Swin-Large backbone (ImageNet-22K), delta1 0.925 and RMS 0.330; on KITTI Eigen, delta1 0.974 and RMS 2.098.Paper · 2022-04-01Perception Prioritized Training of Diffusion ModelsPerception Prioritized (P2) weighting multiplies the simple diffusion MSE loss by 1/(k+SNR(t))^gamma, down-weighting the low-noise steps where the model learns imperceptible detail (k=1, gamma 0.5 or 1). Author-reported FID-50k on FFHQ falls from 7.86 to 6.92 at 1000 steps, and it improves every dataset in the main table.Paper · 2022-03-31BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal TransformersBEVFormer turns six surround-camera images into one bird's-eye-view (BEV) grid using learnable grid queries. Spatial cross-attention samples image features at projected 3D pillar points; temporal self-attention fuses the ego-aligned previous BEV. Author-reported: 56.9 NDS on nuScenes test (V2-99 backbone with extra pre-training), 51.7 NDS on val (ResNet-101).Paper · 2022-03-31BRIO: Bringing Order to Abstractive SummarizationBRIO fine-tunes BART or PEGASUS with cross-entropy on the reference plus a pairwise margin loss that orders the length-normalised log-likelihoods of 16 diverse-beam candidates by their ROUGE. Two trained variants then generate or rerank summaries. Author-reported ROUGE-1: 47.78 on CNN/DailyMail, 49.07 on XSum.Paper · 2022-03-31Exploring Visual Prompts for Adapting Large-Scale ModelsLearns one additive image perturbation (a padding frame, a fixed patch or a random patch) for a frozen model by cross-entropy, then adds it to every test image. On CLIP ViT-B/32 over 12 datasets the author-reported average is 78.2 with the prompt, against 54.2 for zero-shot text prompts and 83.1 for a linear probe. On frozen ImageNet vision models it trails linear probes by about 40 points.Paper · 2022-03-31Multimodal Fusion Transformer for Remote Sensing Image ClassificationMultimodal fusion transformer (MFT) for hyperspectral land-cover classification. A LiDAR, MS, SAR or DSM patch becomes the CLS token, and a CLS-query multihead cross-patch attention fuses it with the HSI patch tokens. Author-reported: highest overall accuracy of the ten methods compared in each of ten dataset settings, e.g. 88.45% on Houston (HSI only) and 97.76% on Trento (HSI only).Paper · 2022-03-30FlowFormer: A Transformer Architecture for Optical FlowFlowFormer turns the all-pairs cost volume of an image pair into compact latent cost tokens, mixes them with alternating within-pixel and across-pixel transformer layers, and decodes flow with a recurrent transformer that queries this cost memory. Author-reported: 1.16 and 2.09 AEPE on Sintel test clean and final, against 1.39 and 2.47 for GMA with warm start.Paper · 2022-03-29LinkBERT: Pretraining Language Models with Document LinksPretrains BERT-style encoders on inputs that put hyperlinked or citation-linked documents in one context, with a three-way document relation prediction loss added to masked language modeling. Author-reported: average MRQA F1 gains over BERT of +4.1 (tiny), +2.6 (base) and +2.5 (large), and new BLURB and MedQA-USMLE state of the art for BioLinkBERT.Paper · 2022-03-28Rethinking Semantic Segmentation: A Prototype ViewReplaces the learnable class weights or queries of segmentation heads with non-learnable prototypes: K sub-cluster centres of training pixel embeddings per class, found by online Sinkhorn clustering and updated by momentum. A pixel takes its nearest prototype's class. Author-reported: +0.6 to +1.2 mIoU over FCN, HRNet, Swin and SegFormer on ADE20K; +1.7 over parametric SegFormer at 847 classes.Paper · 2022-03-28Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary SpaceSplits each transformer feed-forward output into per-value sub-updates. Projected onto the vocabulary, many express human-readable concepts, and top tokens are built by promotion, not elimination. Boosting 10 non-toxic value vectors in GPT-2 cuts toxic continuations by 47% (author-reported). A saturation-based early-exit rule saves 20% of computation (author-reported).Paper · 2022-03-27Long-Tailed Recognition via Weight BalancingTwo-stage long-tailed recognition that uses weight regularisation instead of new losses. Stage 1 trains the backbone with cross-entropy and tuned weight decay; stage 2 retrains the classifier with a class-balanced loss, weight decay and a MaxNorm projection. Author-reported 53.35% class-averaged accuracy on CIFAR100-LT (imbalance factor 100).Paper · 2022-03-27Thin-Plate Spline Motion Model for Image AnimationAnimates a still image with the motion of a driving video, without keypoint labels. Predicts K sets of five keypoints per frame, warps the source with one thin-plate spline field per set plus an affine background, and fuses features with multi-resolution occlusion masks. Author-reported on TaiChiHD: keypoint distance (AKD) 4.57 against 5.41 for the prior MRAA method.Paper · 2022-03-25Stochastic Trajectory Prediction via Motion Indeterminacy DiffusionFrames pedestrian trajectory forecasting as a reverse diffusion. A Transformer denoiser, conditioned on a Trajectron++ encoding of the observed path and neighbours, turns Gaussian noise into a future path over 100 steps. Best-of-20 results: SDD ADE/FDE 7.61/14.30 px; ETH/UCY average 0.21/0.38 m (author-reported).Paper · 2022-03-24CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelsGenerates a textured, normal-mapped mesh from a text prompt with no 3D training data. It optimises a Loop subdivision surface's control vertices, a texture and a normal map through a differentiable renderer, using CLIP similarity on random views, a diffusion-prior loss and a Laplacian regulariser. Author-reported CLIP R-precision 96.7 vs 93.5 for Dreamfields (ViT-B/16 generation and evaluation).Paper · 2022-03-23VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-TrainingVideoMAE pre-trains a plain ViT on unlabelled video by hiding 90% of spacetime cubes with a tube mask shared along time, then reconstructing the hidden pixels from the visible ones. Author-reported, without extra data: ViT-B reaches 81.5% top-1 on Kinetics-400 and 70.8% on Something-Something V2.Paper · 2022-03-22Fast Multi-view Clustering via Ensembles: Towards Scalability, Superiority, and SimplicityFastMICE clusters multi-view data as an ensemble: random groups of views each yield an anchor-based bipartite graph, which is partitioned into a base clustering; M=20 base clusterings are then fused in one bipartite graph. Author-reported: average NMI rank 1.23 over 22 datasets (next best 4.41), near-linear cost, no per-dataset tuning, and only method to finish YTF-400 (398,191 samples).Paper · 2022-03-22Federated Class-Incremental LearningGLFC targets federated class-incremental learning: clients learn new classes over time with a 2,000-sample exemplar memory. Each client uses a gradient-compensated loss and a relation distillation loss; a proxy server rebuilds prototype samples from shared gradients and keeps the best old global model. Author-reported: 4.4% to 15.1% higher average accuracy than adapted baselines.Paper · 2022-03-21Self-Consistency Improves Chain of Thought Reasoning in Language ModelsSelf-consistency replaces greedy decoding in chain-of-thought prompting: sample many reasoning paths from the same prompt, parse each final answer, and return the majority answer. No training or extra models. With 40 samples, GSM8K accuracy rises from 56.5% to 74.4% on PaLM-540B and from 60.1% to 78.0% on code-davinci-002.Paper · 2022-03-21ViM: Out-Of-Distribution with Virtual-logit MatchingPost-hoc OOD score for a trained classifier: the norm of the feature outside a principal subspace (D=1000 for BiT, 512 for ViT) is scaled by a constant fitted on 200k training images, added as a virtual logit, and its softmax probability is the score. Author-reported: average AUROC 90.91% on four OOD sets with BiT-S-R101x1, against 86.62% for the best baseline.Paper · 2022-03-20V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision TransformerV2X-ViT fuses LiDAR feature maps from connected vehicles and roadside infrastructure with a Transformer. It stacks heterogeneous multi-agent self-attention and multi-scale window attention, plus a delay-aware encoding. Author-reported: noisy-setting AP@0.7 of 0.614 vs 0.402 for no fusion on the authors' simulated V2XSet.Paper · 2022-03-19ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical ReasoningChartQA benchmark: 9.6K human-written and 23.1K machine-generated questions on real-world charts. Two baselines, VisionTaPas and VL-T5, combine chart-image features with a table extracted from the chart. Best author-reported ChartQA test accuracy: 45.52% with extracted tables, 61.84% with gold tables.Paper · 2022-03-18Spinning-enabled Wireless Amphibious Origami MillirobotA magnetically driven millirobot made of a Kresling origami cylinder with a magnetic plate. A rotating field makes it roll, flip or spin on ground and swim in water. Folding works as a liquid pump and spinning as a suction grip. Author-reported: 81.2 mm/s swimming at 10 mT and 30 Hz with a frontal hole and radial cuts, and a 23.5 mm jump at 40 mT and 120 degrees.Paper · 2022-03-17ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech DetectionGenerates 274k toxic and benign statements about 13 minority groups by demonstration-based prompting of GPT-3. ALICE, a constrained beam search, steers generation against a toxicity classifier to make harder examples. Fine-tuning HateBERT or RoBERTa on the data raises AUC on three human-written implicit-hate sets (author-reported).Paper · 2022-03-16Decoupled Knowledge DistillationSplits the classic KD loss into a target-class term (TCKD) and a non-target-class term (NCKD), drops the teacher-confidence factor from NCKD and weights both terms separately (alpha, beta). Logit-only, so no extra cost over KD. Author-reported: ResNet32x4 to ResNet8x4 on CIFAR-100 reaches 76.32% top-1 vs 73.33% for KD.Paper · 2022-03-16Memorizing TransformersAdds a non-differentiable external memory of past (key, value) pairs to one attention layer of a decoder-only transformer. Each query retrieves its top-k stored pairs by approximate kNN and blends that result with local attention through a learned per-head gate. Perplexity falls on arXiv, PG-19, C4, GitHub and Isabelle as memory grows to 65K tokens (author-reported).Paper · 2022-03-16Unsupervised Semantic Segmentation by Distilling Feature CorrespondencesSTEGO freezes a self-supervised DINO ViT and trains a small head so that pixel codes keep the feature correspondences of the backbone (cosine similarity, self, nearest-neighbour and random pairs), then clusters the codes into classes. Author-reported: 28.2 unsupervised mIoU on CocoStuff-27 (+14 over PiCIE+H) and 21.0 on Cityscapes-27 (+9).Paper · 2022-03-15MotionCLIP: Exposing Human Motion Generation to CLIP SpaceMotionCLIP trains a transformer auto-encoder on SMPL motion so each latent vector also lands near CLIP's embedding of the motion's label and of a rendered frame. A text prompt, encoded by frozen CLIP, decodes directly into motion with no text-motion training. Author-reported: 76.7% user preference over JL2P in-domain; 40.9% top-1 on BABEL-60 vs 41.14% for 2s-AGCN.Paper · 2022-03-11Masked Visual Pre-training for Motor ControlPre-trains a ViT-S masked autoencoder on about 700k in-the-wild images of hands and objects, freezes it, and trains small PPO controllers on its features for pixel-based robot tasks, with no per-task fine-tuning. Author-reported: beats a supervised ImageNet encoder on 7 of 8 tasks, by up to 80 points of success rate, and sometimes matches the oracle-state policy.Paper · 2022-03-10Backbone is All Your Need: A Simplified Architecture for Visual Object TrackingSimTrack feeds exemplar and search patch tokens into one ViT backbone, so attention mixes the two images in every layer. It drops the Siamese correlation and transformer head. A foveal window adds tokens cropped from the exemplar centre. ViT-B/16 reaches 69.3% AUC on LaSOT and 54.8% on TNL2K, author-reported.Paper · 2022-03-08Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-LabelsU2PL trains segmentation on few labeled and many unlabeled images. High-entropy teacher pixels are dropped from the pseudo-label loss but kept as negatives in per-class FIFO queues for a pixel-level InfoNCE loss. The dropped share starts at 20% and shrinks linearly. ResNet-101 with DeepLabv3+ on VOC 2012, 1/16 classic split: 67.98 mIoU vs 57.00 for PC2Seg. Author-reported.Paper · 2022-03-07Kubric: A scalable dataset generatorKubric is an open-source Python framework that drives PyBullet physics and Blender Cycles ray tracing, one random scene per worker, scaled across machines, with per-pixel and object annotations. Its dataset use cases report author-run results, e.g. RAFT pretrained on Kubric data reaches AEPE 4.82 on KITTI-15, versus 7.63 for FlyingChairs and 4.66 for AutoFlow.Paper · 2022-03-06GeoDiff: a Geometric Diffusion Model for Molecular Conformation GenerationGeoDiff generates 3D atom coordinates for a molecular graph by reversing a diffusion process. Its reverse kernels are roto-translation equivariant and it starts from a centre-of-mass-free Gaussian, so the sample density is invariant. On GEOM-Drugs (author-reported), mean COV-R rises to 89.13% from 67.16% for the best baseline (GeoMol).Paper · 2022-03-03NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth EstimationMonocular depth decoder that runs fully-connected CRF energy optimisation inside non-overlapping 7x7-patch windows (shifted every other step), with multi-head attention as the pairwise potential, on a Swin encoder. Author-reported: KITTI Eigen Abs Rel 0.052 (from 0.058 for AdaBins), NYUv2 Abs Rel 0.095, Matterport3D Abs Rel 0.0906.Paper · 2022-03-02A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and ChallengesSurvey of aspect-based sentiment analysis. Proposes a taxonomy by the four sentiment elements (aspect term, aspect category, opinion term, polarity), groups single and compound ABSA tasks with their modeling paradigms, and reviews pre-trained language models, cross-domain and cross-lingual transfer, and open challenges. Reports no new experiments.Paper · 2022-03-02A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open ProblemsSurvey of offline RL. It proposes a taxonomy of method modifications (policy constraints, importance sampling, regularization, uncertainty, model-based, one-step, imitation, trajectory optimization), reviews algorithms in one notation, reviews D4RL, RL Unplugged and off-policy evaluation, and lists open problems. It proposes no new method; its heat map aggregates collected scores.Paper · 2022-03-02Class Re-Activation Maps for Weakly-Supervised Semantic SegmentationReCAM: after a ResNet-50 is trained with BCE and CAMs are extracted, a second FC layer is trained on CAM-masked features with softmax cross-entropy (loss = BCE + λ·SCE); new CAMs from that model are the seeds. VOC seeds: CAM 48.8 → ReCAM 54.8 mIoU; with IRN refinement 66.3 → 70.9 (author-reported).Paper · 2022-02-28StrongSORT: Make DeepSORT Great AgainStrongSORT upgrades DeepSORT with a YOLOX-X detector, BoT appearance features, EMA feature updates, camera compensation, noise-adaptive Kalman updates and one global assignment. Two add-ons: AFLink links tracklets without appearance, and GSI smooths interpolated boxes with Gaussian-process regression. Author-reported: StrongSORT++ reaches 64.4 HOTA and 79.5 IDF1 on MOT17 test.Paper · 2022-02-26An End-to-End Transformer Model for Crowd LocalizationCLTR casts crowd localization as direct set prediction: a ResNet-50 feature map (stride 32) feeds a 6+6-layer transformer with 500 learned queries that output head points and confidences. Training matches predictions with a Hungarian matcher plus a kNN-context cost (KMO). Author-reported: best F1 on UCF-QNRF (80.97) and ShanghaiTech Part A at sigma=4 (43.2); NWPU test F1 68.5, behind TopoCount.Paper · 2022-02-25Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Analysis paper: replacing the labels of in-context demonstrations with uniformly random labels costs only 0-5% absolute on 26 classification and multi-choice datasets across 12 model/inference combinations (774M to 175B, incl. GPT-3). Ablations show the label space, the input text distribution and the input-label pair format drive the gains, not correct pairings.Paper · 2022-02-23COLD Decoding: Energy-based Constrained Text Generation with Langevin DynamicsCOLD decoding treats the output as a soft sequence of logits, defines an energy from weighted constraint terms, samples with Langevin dynamics, then snaps each position to a top-k token of a GPT2-XL LM. No fine-tuning. Author-reported: best overall coherence on abductive reasoning and highest keyword coverage on CommonGen.Paper · 2022-02-22A New Generation of Perspective API: Efficient Multilingual Character-level TransformersUTC is a token-free, byte-level Charformer encoder-decoder for multilingual toxic-comment scoring in Perspective API. It is pretrained with span denoising on Perspective-internal text and mC4, then fine-tuned with a pooled regression head. Author-reported: 0.910 (102M) and 0.919 (268M) average AUC-ROC on production multilingual data; 0.9367 (268M) on Kaggle JMTCC.Paper · 2022-02-22GroupViT: Semantic Segmentation Emerges from Text SupervisionTrains a ViT-S with hierarchical grouping blocks on image-text pairs using only contrastive losses, no masks. Output segments are matched to label-name embeddings for zero-shot semantic segmentation. Author-reported zero-shot mIoU: 52.3 on PASCAL VOC 2012 and 22.4 on PASCAL Context, with no fine-tuning.Paper · 2022-02-21Fine-Tuning can Distort Pretrained Features and Underperform Out-of-DistributionFine-tuning all layers of a pretrained network can give worse out-of-distribution accuracy than linear probing when pretrained features are good and the shift is large. A two-layer linear theory explains this by feature distortion. Linear probing then fine-tuning (LP-FT) has the best average ID and OOD accuracy on 10 shifts; author-reported OOD average 68.9 vs 59.3 for fine-tuning.Paper · 2022-02-21Transformer Quality in Linear TimeA gated attention unit (GAU) gates one small softmax-free attention head, so the attention can be weak. FLASH approximates it in linear time: exact attention inside chunks, linear attention across chunks. Author-reported: Transformer++-level perplexity at 512 to 8K context, with training speedups up to 4.9x on Wiki-40B, 4.8x on C4 MLM and 12.1x on PG-19.Paper · 2022-02-16ActionFormer: Localizing Moments of Actions with TransformersActionFormer labels every time step of an untrimmed video with an action class and the distances to the action's start and end, in one pass and without proposals or anchors. A Transformer encoder with local self-attention builds a feature pyramid; light 1D conv heads predict the outputs. Author-reported: 71.0% mAP at tIoU 0.5 on THUMOS14 (I3D).Paper · 2022-02-14Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle RacingSurvey of autonomous racing cars up to end-2021 (233 papers). Sorts software into perception, planning (global, local, behavioural), control (classic, MPC, learning, drifting) and end-to-end work, then reviews race platforms (1:43 to real race cars) and nine open challenges. No experiments of its own; numbers are those of the surveyed works.Paper · 2022-02-14How Do Vision Transformers Work?Argues that self-attention in vision acts as a learned spatial smoothing that flattens loss landscapes, and that ViTs struggle on small data because their losses are non-convex. Proposes AlterNet, a ResNet with stage-end Conv blocks partly replaced by local attention, reported to beat CNNs on CIFAR-100. Author-reported; numbers appear only in figures.Paper · 2022-02-13A Contrastive Framework for Neural Text GenerationSimCTG adds a contrastive loss to GPT-2 fine-tuning that makes token representations discriminative. Contrastive search picks each next token from the top-k candidates by model probability minus a penalty for similarity to the context. On WikiText-103 it is best or tied-best on diversity, MAUVE and coherence (author-reported).Paper · 2022-02-11Compute Trends Across Three Eras of Machine LearningFits log-linear trends to the training compute of 123 milestone ML systems from 1952 to 2022. Compute doubled about every 21 months before 2010, about every 6 months from 2010 to 2022, and a separate large-scale trend since late 2015 doubled about every 10 months. Author-reported estimates with bootstrap intervals.Paper · 2022-02-11The Shapley Value in Machine LearningSurvey of the Shapley value in machine learning. It covers the cooperative-game definition and its four axioms, sampling and regression approximations, and applications to feature selection, data valuation, explainability, federated learning, multi-agent RL and ensembles. It has no experiments and no code; its only worked numbers come from a 3-player example.Paper · 2022-02-08Survey of Hallucination in Natural Language GenerationSurvey that defines hallucination in NLG as output that is nonsensical or unfaithful to the source, splits it into intrinsic (contradicts the source) and extrinsic (unverifiable from the source), and catalogues causes, metrics (statistical, IE, QA, NLI, classifier, LM-based) and mitigations across summarization, dialogue, GQA, data-to-text, MT, vision-language and LLMs.Paper · 2022-02-07Context Autoencoder for Self-Supervised Representation LearningContext autoencoder (CAE) pretrains a ViT by hiding 50% of the patches. An encoder sees only the visible ones, a cross-attention regressor predicts the hidden patches' encoder representations, and a decoder reconstructs discrete tokens from them. Author-reported: ViT-B at 1600 epochs gives 83.9 fine-tuning, 77.1 attentive probing and 50.2 ADE20K mIoU.Paper · 2022-02-07OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning FrameworkOFA is one encoder-decoder Transformer for vision, language and cross-modal tasks. Each task is an instruction plus inputs that yields tokens (text, image codes or box bins). Pretrained on 20M public image-text pairs and unimodal data, it reports SOTA on VQA test-std (82.0), SNLI-VE and COCO CIDEr-optimised captioning (154.9). Author-reported; unimodal results are competitive, not always best.Paper · 2022-02-07data2vec: A General Framework for Self-supervised Learning in Speech, Vision and LanguageOne self-distillation recipe for speech, images and text: a student regresses the average of the top K normalised blocks of an EMA teacher that sees the unmasked input, so targets are contextualised and need no tokenizer. Single-model ImageNet-1K top-1 of 84.2 (ViT-B) and 86.6 (ViT-L), and lower WER in Libri-light low-resource setups (author-reported).Paper · 2022-02-05Handling Distribution Shifts on Graphs: An Invariance PerspectiveFormulates out-of-distribution generalization for node-level prediction on graphs and proposes EERM: adversarial graph editors create virtual environments from one observed graph, and the GNN minimises the mean plus variance of their risks. Author-reported gains on synthetic spurious-feature shifts, Twitch and Facebook-100 transfers, and Elliptic and OGB-Arxiv temporal splits.Paper · 2022-02-04BC-Z: Zero-Shot Task Generalization with Robotic Imitation LearningBC-Z trains one image-to-action policy for a real robot on 100 manipulation tasks (25,877 demonstrations, including human corrections during rollouts). The task is given as a language embedding or a human-video embedding, injected through FiLM layers. Author-reported: language commands reach 32% average success on held-out tasks, video commands 4%.Paper · 2022-02-03CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series ForecastingPretrains a dilated-TCN encoder with two contrastive branches: a MoCo loss on a mixture-of-AR-experts trend branch, and amplitude and phase losses on a learnable Fourier seasonal branch. A ridge regressor forecasts from the frozen features. Author-reported: 21.3% lower average MSE than the best feature-based baseline (TS2Vec) on five multivariate benchmarks.Paper · 2022-02-03ETSformer: Exponential Smoothing Transformers for Time-series ForecastingReplaces Transformer self-attention with exponential smoothing attention (learned decay over lags, FFT convolution) for growth and frequency attention (top-K Fourier bases) for seasonality. The decoder forecasts level, growth and season separately. Author-reported: lowest MSE in 22 of the 24 multivariate rows shown, six datasets.Paper · 2022-02-03The Disagreement Problem in Explainable Machine Learning: A Practitioner's PerspectiveDefines disagreement between post hoc feature-attribution explanations and proposes six metrics: feature, rank, sign and signed-rank agreement on top-k features, plus rank correlation and pairwise rank agreement on chosen features. Six methods disagree often on tabular, text and image models. In an online study, practitioners mostly resolved conflicts with ad hoc heuristics. Author-reported.Paper · 2022-01-31Trajectory balance: Improved credit assignment in GFlowNetsTrajectory balance (TB) trains a GFlowNet on whole sampled trajectories: a squared log-ratio between log Z plus summed forward log-probabilities and log reward plus summed backward log-probabilities. Any global minimiser samples in proportion to reward. Author-reported: faster convergence and more diverse samples than flow matching and detailed balance.Paper · 2022-01-28BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationBLIP pre-trains one image-text model that can match, encode and caption. A captioner writes synthetic captions for noisy web images and a filter drops mismatched pairs (CapFilt). Author-reported state of the art on COCO and Flickr30K retrieval, COCO and NoCaps captioning, and VQA.Paper · 2022-01-28Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsPrompts a language model with few-shot exemplars whose answers include a written chain of intermediate reasoning steps, with no finetuning. Above roughly 100B parameters this lifts arithmetic, commonsense and symbolic reasoning accuracy; PaLM 540B reaches 56.9% on GSM8K (standard prompting 17.9%), author-reported.Paper · 2022-01-24Text and Code Embeddings by Contrastive Pre-TrainingTrains text and code embedding encoders by contrastive pre-training on unlabeled paired data (neighbouring web text; docstring-function pairs), with in-batch negatives, large batches and GPT/Codex initialisation. Author-reported: best linear-probe accuracy on 7 SentEval tasks, strong single-embedding search on MSMARCO, NQ and TriviaQA, and a 20.8% relative gain over GraphCodeBERT on CodeSearchNet.Paper · 2022-01-23A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-LocalizationFSRA is a ViT-S baseline for drone-to-satellite geo-localization. It sorts patch tokens by mean activation, splits them into n=3 regions, pools each region and aligns drone and satellite regions with ID and triplet losses, with no extra labels. Author-reported on University-1652: 84.51% R@1 and 86.71% AP at k=3 satellite repeats.Paper · 2022-01-21GreaseLM: Graph REASoning Enhanced Language Models for Question AnsweringGreaseLM answers multiple-choice questions with a pretrained language model and a graph neural network over a knowledge-graph subgraph linked to the question. An interaction token and an interaction node pass information between the two in M fusion layers. Author-reported: 74.2 IHtest on CommonsenseQA, 84.8 on OpenbookQA test, 38.5 on MedQA-USMLE.Paper · 2022-01-21Point-NeRF: Point-based Neural Radiance FieldsPoint-NeRF models a radiance field as a cloud of neural points, each with a feature and a confidence. A pre-trained MVS-style network predicts the initial points, which are rendered by differentiable ray marching and then pruned and grown per scene. Author-reported: 33.31 PSNR on NeRF Synthetic after 200K steps and 30.71 after 20K steps, versus 31.01 for NeRF.Paper · 2022-01-20From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AISystematic review of 312 papers (2014-2020, 12 AI/ML conferences) that introduce an explanation method. Proposes twelve explanation-quality properties (Co-12) and catalogues 29 automated quantitative evaluation methods, each mapped to the properties it can check. Author-reported: 33% of these papers evaluate only with anecdotal evidence, 58% quantitatively, 22% with user studies.Paper · 2022-01-20LaMDA: Language Models for Dialog ApplicationsDecoder-only dialog Transformers from 2B to 137B parameters, pre-trained on 1.56T words of public dialog and web text. Crowd-labelled fine-tuning filters and re-ranks replies for quality and safety, and teaches the model to query a search, calculator and translator tool for grounding. Author-reported at 137B: 92.3% sensibleness, 95.2% safety, 73.2% groundedness.Paper · 2022-01-19Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and ComparisonsFramework for total uncertainty in physics-informed networks and DeepONets: Bayesian methods and ensembles, GAN and polynomial-chaos priors, six metrics, and post-training calibration, compared on prototype PDE, SPDE and operator-learning problems. Cheaper methods (LA, MFVI, SEns) trade off well; a small left-out set reduces calibration error.Paper · 2022-01-18TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series DataTranAD detects and diagnoses anomalies in multivariate time series with a transformer encoder and two decoders. A two-phase pass conditions the second reconstruction on the first one's error, and adversarial two-decoder training amplifies small deviations; MAML is added for limited data. Author-reported: up to 17% higher F1 than baselines and up to 99% lower training time.Paper · 2022-01-17A Literature Survey of Recent Advances in ChatbotsLiterature survey of chatbots built with AI and NLP, from 316 articles (2007-2021) found through publisher databases, arXiv, Google Scholar and reference lists. It contrasts rule-based, retrieval and generative (Seq2Seq, Transformer) designs, lists OpenSubtitles, Cornell and DailyDialog, and defines BLEU, METEOR, TER, F1 and perplexity. It finds no shared evaluation framework. No code.Paper · 2022-01-17Towards a Cleaner Document-Oriented Multilingual Crawled CorpusRebuilds the OSCAR multilingual web corpus from one Common Crawl dump so that each document gets one language label instead of one per line. It adds length-based head/tail and short-line filters plus non-destructive quality annotations. Author-reported: mid- and high-resource corpus sizes stay near OSCAR 21.09, several low-resource ones shrink.Paper · 2022-01-14A Survey of Controllable Text Generation using Transformer-based Pre-trained Language ModelsSurvey of controllable text generation (CTG) with transformer-based pre-trained language models. Sorts methods into fine-tuning, retraining/refactoring and decoding-time post-processing, and covers seven task families, PLM types and evaluation metrics. It runs no experiments of its own; it concludes that dedicated CTG benchmarks and reliable automatic evaluation are still missing.Paper · 2022-01-10A ConvNet for the 2020sStarts from a ResNet-50 and modernizes it step by step with Transformer-era training recipes and design choices (7x7 depthwise convs, inverted bottlenecks, LayerNorm, GELU, patchify stem), ending in a pure ConvNet, ConvNeXt. Author-reported: ConvNeXt-T 82.1% ImageNet-1K top-1 vs Swin-T 81.3%; ConvNeXt-XL (22K pre-training, 384px) 87.8%; matches or beats Swin on COCO and ADE20K.Paper · 2022-01-10Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural NetworksSurvey of recent CNN regularizers, grouped into input methods (data augmentation such as Cutout, Mixup, CutMix and AutoAugment), internal methods (Dropout variants, Shake-Shake, ShakeDrop, DropBlock) and label methods (label smoothing, TSLA, SLS, JoCoR). It compiles published error rates on CIFAR-10/100, SVHN and ImageNet. It runs no experiments of its own.Paper · 2022-01-07Generalized Category DiscoveryLabels every unlabelled image when it may come from known or new classes and the class count is unknown. A DINO-pretrained ViT-B/16 is tuned with contrastive losses, then semi-supervised k-means clusters its [CLS] features; k comes from a labelled-accuracy search. Author-reported "All" accuracy: CIFAR10 91.5, CIFAR100 73.0, ImageNet-100 74.1, CUB 51.3, Stanford Cars 39.0, Herbarium19 35.4.Paper · 2022-01-05Rethinking Depth Estimation for Multi-View Stereo: A Unified RepresentationUniMVSNet codes each pixel's depth as its best depth-hypothesis interval plus a proximity value, trained with a sigmoid per hypothesis and a relative-error Unified Focal Loss, then decoded by an offset. In a 3-stage coarse-to-fine network it reports first place on DTU Overall (0.315 mm) and on Tanks and Temples (author-reported).Paper · 2022-01-04A Transformer-Based Siamese Network for Change DetectionChangeFormer is a Siamese network for change detection in remote-sensing image pairs. A hierarchical transformer encoder with reduced-sequence attention reads both images; per-scale difference modules feed a light MLP decoder that predicts a change mask. Author-reported: F1 90.40 on LEVIR-CD (BIT: 89.31) and 86.67 on DSIFN-CD (BIT: 69.26).Paper · 2022-01-02Splicing ViT Features for Semantic Appearance TransferTrains a small U-Net on one structure/appearance pair to repaint the structure image with the look of semantically related objects. Losses use frozen DINO ViT-B/8 features: the deepest CLS token for appearance, deepest-layer key self-similarity for structure, and a key identity term. No adversarial training. Author-reported AMT preference for Splice over STROTSS (79.0%) on Wild-Pairs.Paper · 2021-12-31Machine learning based disease diagnosis: A comprehensive reviewSystematic review of machine learning and deep learning for disease diagnosis, 2012 to 2021. It searches Scopus and Web of Science, runs a bibliometric analysis of 1,216 papers, and summarizes 102 full-text studies by disease. Reports that CNNs are the most used model. No experiments; the accuracies it lists are the reviewed papers' own.Paper · 2021-12-29A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language ModelTwo-stage open-vocabulary segmentation: a MaskFormer trained on seen classes proposes masks, and a frozen CLIP ViT-B/16 labels each masked crop against text prompts. Author-reported zero-shot hIoU on COCO-Stuff 37.8 (FCN 20.9) and on Pascal VOC 77.5 (best prior without self-training 39.7), both without self-training.Paper · 2021-12-29StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2StyleGAN-V extends StyleGAN2 to continuous-time video. A conv1d network over sparse noise produces acyclic sinusoidal motion codes that can be queried at any frame time, and the discriminator scores 2 to 4 random frames per clip jointly. Author-reported: lowest FVD16 on FaceForensics, SkyTimelapse and UCF101 at 256 squared (not RainbowJelly), and training at 1024 squared on MEAD.Paper · 2021-12-23SLIP: Self-supervision meets Language-Image Pre-trainingAdds a SimCLR-style image self-supervision loss, weight 1.0, to CLIP's image-text contrastive loss on one shared Vision Transformer. On YFCC15M, ViT-B/16 reaches 42.8% zero-shot and 72.1% linear ImageNet accuracy, against 37.6% and 66.5% for the authors' CLIP run (author-reported).Paper · 2021-12-22Scaling Open-Vocabulary Image Segmentation with Image-Level LabelsOpenSeg predicts class-agnostic mask proposals, then aligns each caption word to one or a few masks with a grounding loss, so it can learn segmentation from image captions as well as masks. Trained on COCO and Localized Narratives with an EfficientNet-B7 backbone, the author-reported PASCAL-20 mIoU is 72.2 with prompt engineering, 19.9 above the LSeg ViT-L result it is compared with.Paper · 2021-12-22YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehiclesModifies YOLOv5 for small objects (traffic cones seen from an autonomous racing car): DenseNet or ResNet50 backbone, FPN or bi-FPN neck, and an extra higher-resolution feature map that replaces the lowest-resolution one. Author-reported: 5.5 to 7.7 percent relative mAP@0.5 gain on small objects across S to X scales, for 0.9 to 3.7 ms more inference time.Paper · 2021-12-20ScanQA: 3D Question Answering for Spatial Scene UnderstandingScanQA answers free-form questions about a 3D indoor scan and points to the objects the question refers to. A VoteNet detector and a GloVe/LSTM question encoder feed transformer fusion layers that drive localization, object-class and answer heads. Author-reported: EM@1 23.45 with objects and 20.90 without, above every listed 2D or pipeline baseline.Paper · 2021-12-19M2DGR: A Multi-sensor and Multi-scenario SLAM Dataset for Ground RobotsDataset of 36 ROS-bag sequences (about 1 TB) from a ground robot with six fish-eye cameras, a sky camera, a 32-beam LiDAR, IMU, GNSS-RTK, thermal and event cameras. Indoor, outdoor, lift and darkness scenes, with motion-capture or laser ground truth. Author-reported: the evaluated SLAM systems perform poorly in some scenarios.Paper · 2021-12-17Align and Prompt: Video-and-Language Pre-training with Entity PromptsALPRO pre-trains a sparse-frame video-text encoder on 5.5M web pairs with a video-text contrastive loss that aligns unimodal features, and prompting entity modeling, which predicts soft entity labels for random video crops from a frozen CLIP-style prompter. Fine-tuned, it reports 33.9 R@1 on MSRVTT 1k and 42.1 MSRVTT-QA accuracy (author-reported).Paper · 2021-12-16Learning To Retrieve Prompts for In-Context LearningTrains a small dense retriever (two BERT-base encoders) to pick in-context examples. Candidates are labelled by how likely a scoring LM (GPT-Neo 2.7B) finds the gold output given each one: top-5 positives, bottom-5 hard negatives. Author-reported dev results vs best baseline: 31.9 vs 26.0 (BREAK, LF-EM), 64.2 vs 57.0 (MTOP, EM), 54.3 vs 51.4 (SMCalFlow, EM).Paper · 2021-12-15LongT5: Efficient Text-To-Text Transformer for Long SequencesLongT5 extends the T5 encoder to long inputs with Transient Global (TGlobal) attention: local sliding-window attention plus per-block global tokens built on the fly, with no extra side inputs. It is pretrained with PEGASUS-style principle-sentence generation. Author-reported: state of the art on arXiv, PubMed, BigPatent, MediaSum and TriviaQA with up to 16k input tokens.Paper · 2021-12-15On the Convergence and Robustness of Adversarial TrainingScores how well each PGD adversarial example solves the inner maximisation (FOSC) and trains early epochs on weaker examples, later on stronger ones, via a linearly falling FOSC threshold. Author-reported CIFAR-10 white-box PGD-10 accuracy with an 8-layer CNN: 48.06% vs 44.39% for standard PGD training.Paper · 2021-12-13Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI PerspectiveSurvey of data collection and data quality for deep learning from a data-centric view: acquisition, labeling, validation, cleaning, poisoning defenses, robust training under noisy or missing features and labels, and fairness measures with mitigation before, during and after training. Reports no experiments of its own; its taxonomy and the procedures it describes are the content.Paper · 2021-12-13GLaM: Efficient Scaling of Language Models with Mixture-of-ExpertsGLaM is a decoder-only Transformer whose every other feed-forward layer is a sparse mixture of 64 experts, with two experts per token. Its 1.2T-parameter model (96.6B active) beats GPT-3 (175B) on average over 29 zero-, one- and few-shot tasks at about half the inference FLOPs and a third of the training energy (author-reported).Paper · 2021-12-10BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene RenderingBungeeNeRF trains a NeRF on city scenes from satellite to street level. It starts with a small base block for the most distant scale, then adds one residual block per stage, each supervised on all scales up to its own, so one model gives several levels of detail. Author-reported: average PSNR 23.5 and 23.6 on two city scenes at the baselines' budget, and 24.5 and 24.4 with longer training.Paper · 2021-12-10Deep ViT Features as Dense Visual DescriptorsUses keys from a self-supervised DINO ViT as dense patch descriptors. Zero-shot k-means clustering with saliency voting gives co-segmentation and part co-segmentation; log-binned keys with mutual nearest neighbours give point correspondences. No training. Author-reported: 79.5 J_m on PASCAL-CO co-segmentation, above the best unsupervised baseline (75.8).Paper · 2021-12-10More Control for Free! Image Synthesis with Semantic Diffusion GuidanceAdds language, reference-image or combined guidance to a pretrained unconditional DDPM at sampling time, without retraining it. Each denoising step is shifted along the gradient of a CLIP-based similarity; CLIP is finetuned on noised images without text labels. Author-reported FFHQ FID: 14.37 for image guidance (ILVR 17.15), 28.38 for text guidance.Paper · 2021-12-09Neural Descriptor Fields: SE(3)-Equivariant Object Representations for ManipulationMaps each 3D point to a descriptor taken from an occupancy network's layer activations, made SE(3)-equivariant with Vector Neurons and trained without keypoint labels. Poses are found by matching query-point descriptors to a demonstration. Author-reported: 0.58-0.91 simulated success on unseen objects, versus 0.00-0.45 for Dense Object Nets.Paper · 2021-12-09Plenoxels: Radiance Fields without Neural NetworksFits a sparse voxel grid whose cells store opacity and degree-2 spherical harmonic colour coefficients, using NeRF's volume rendering, trilinear interpolation, an MSE and total-variation loss, coarse-to-fine pruning and RMSProp, with no neural network. Author-reported: 31.71 PSNR on the 8 NeRF synthetic scenes, about 11 minutes per scene versus about a day for NeRF.Paper · 2021-12-08Ethical and social risks of harm from Language ModelsA taxonomy of 21 ethical and social risks of large language models in six areas: discrimination, information hazards, misinformation, malicious use, conversational-agent harms, and automation and environment. For each risk it gives the mechanism, the harm types and the point of origin. No experiments or code: a checklist for risk assessment, not a method to implement.Paper · 2021-12-08FLAVA: A Foundational Language And Vision Alignment ModelFLAVA pretrains one transformer family (image, text and fusion encoders) on 70M public image-text pairs plus unpaired ImageNet and text. It mixes contrastive, masked-multimodal, masked-image, masked-language and image-text-matching losses. Author-reported macro average over 35 tasks: 75.85, vs 66.78 for CLIP ViT-B/16 trained on 400M pairs.Paper · 2021-12-08Improving language models by retrieving from trillions of tokensConditions a decoder-only Transformer on chunks retrieved by frozen BERT nearest-neighbour search from a 2-trillion-token database, read through chunked cross-attention. A 7.5B model outperforms Jurassic-1 and Gopher on a majority of Pile test sets; retrieval can also be added to a pretrained model (RETRO-fitting). Author-reported.Paper · 2021-12-08Neural Network Quantization for Efficient Inference: A SurveySurvey of neural network quantization for inference: number formats, quantization-aware training versus post-training quantization, and integer/fixed-point, binary, ternary and mixed-precision schemes, with comparison tables and open directions. Proposes no new method or experiments; the tables report numbers from the cited works (CIFAR-10, ImageNet).Paper · 2021-12-07Soft Robots Modeling: a Structured OverviewA structured review of how soft robots are modeled. It sorts the approaches into continuum mechanics, geometric (functional and constant-curvature), discrete and data-driven families, gives their governing equations, and compares accuracy, cost and suitability for control and simulation. It proposes no new method and reports no experiments.Paper · 2021-12-06Label-Efficient Semantic Segmentation with Diffusion ModelsUses decoder activations of a pretrained DDPM, taken at three diffusion steps and five UNet blocks, as per-pixel features. An ensemble of ten small MLPs is trained on a few annotated images and majority-votes each pixel's label. Author-reported mean IoU is the best of the compared methods on all six datasets, e.g. 49.4 on Bedroom-28 and 59.1 on FFHQ-34.Paper · 2021-12-06Text2Mesh: Text-Driven Neural Stylization for MeshesText2Mesh stylizes a fixed input mesh from a text prompt. A small MLP maps each vertex to an RGB color and a bounded offset along its normal. CLIP similarity between the text and augmented multi-view renders trains it, with no pretrained 3D generator or dataset. Authors report a user-study score of 3.90 against 2.83 for an extended VQGAN-CLIP baseline.Paper · 2021-12-02The Surprising Effectiveness of Representation Learning for Visual ImitationVINN decouples visual imitation in two stages: a ResNet-50 encoder, initialised from ImageNet and fine-tuned with BYOL on the demonstration frames, then a distance-weighted average of the actions of the k nearest demonstration frames. On a real door-opening task with a Hello Robot Stretch it reached 80% success (30 rollouts), against 53.3% for BC on frozen representations and 0% for end-to-end BC.Paper · 2021-12-02Zero-Shot Text-Guided Object Generation with Dream FieldsDream Fields optimises a NeRF so that its renders from random 360-degree camera angles score high against a text caption under a frozen CLIP model, with no 3D training data. Transmittance regularisation, background augmentation, scene bounds and a residual MLP keep the objects coherent. Author-reported CLIP R-Precision is 68.3 at 20K iterations with 252-pixel renders.Paper · 2021-12-01CLIPstyler: Image Style Transfer with a Single Text ConditionStylises a content image from a text prompt alone. For each image it trains a small U-Net against CLIP losses: a whole-image directional loss and a patch-wise directional loss over 64 perspective-augmented crops with threshold rejection. Author-reported CLIP score 0.2515 (best of seven methods) and user-study style score 3.78.Paper · 2021-12-01SegDiff: Image Segmentation with Diffusion Probabilistic ModelsAdapts a DDPM to segmentation. The noise predictor adds image features (an RRDB encoder) to the features of the noisy mask after the first convolution and is trained end-to-end without a pretrained backbone. At inference, several sampled masks are averaged and thresholded. Author-reported: Cityscapes per-object mIoU 76.44 (expansion setting), Vaihingen F1 95.14, MoNuSeg Dice 81.59.Paper · 2021-11-26AI and the Everything in the Whole Wide World BenchmarkPosition paper arguing that a few "general" AI benchmarks (ImageNet, GLUE, SuperGLUE) lack construct validity as measures of general ability: their tasks are arbitrary, de-contextualised and socially situated, and the field over-reads them. It proposes scoped benchmarks plus test suites, audits, error and disaggregated analysis, ablations and model-property measures. It reports no experiments.Paper · 2021-11-26GMFlow: Learning Optical Flow via Global MatchingGMFlow estimates optical flow by matching every pixel of frame one against all pixels of frame two with a Transformer-enhanced feature softmax, not stacked iterative refinements. Author-reported: Things validation EPE 2.80 with one refinement vs 4.25 for RAFT with 31; Sintel test clean 1.74 vs 1.94. Worse than RAFT on KITTI test (F1-all 9.32 vs 5.10).Paper · 2021-11-24Adaptive Fourier Neural Operators: Efficient Token Mixers for TransformersAFNO replaces self-attention token mixing with a Fourier-domain operator: a 2D FFT, block-diagonal channel mixing with weights shared across tokens, soft-thresholding of frequencies, and an inverse FFT. Cost is quasi-linear in token count. Author-reported: matches self-attention on inpainting and beats efficient attention variants on Cityscapes at 1024x1024 (80.9 mIoU with SegFormer-B3).Paper · 2021-11-24NAM: Normalization-based Attention ModuleNAM is an attention module that reuses batch-norm scale factors as channel importance weights, so it adds no fully connected or convolutional layers, and adds an L1 penalty on those factors to suppress weak weights. Author-reported: ResNet-50 on CIFAR-100 top-1 error 19.09% (channel-only NAM) vs 19.44% for CBAM; MobileNet V2 on ImageNet 29.34% vs 29.74%. Released code: channel part only.Paper · 2021-11-23U-shape Transformer for Underwater Image EnhancementUnderwater image enhancement with a U-shaped generator. A channel-wise transformer (CMSFFT) replaces the skip connections and a spatial transformer (SGFMT) sits at the bottleneck. Trained as a GAN with added LAB, LCH and perceptual losses. Also introduces LSUI, 4279 real underwater image pairs. Author-reported PSNR 24.16 and SSIM 0.93 on Test-L400, the highest of the 10 compared methods.Paper · 2021-11-18ClipCap: CLIP Prefix for Image CaptioningMaps a frozen CLIP image embedding to a short prefix of GPT-2 input embeddings, trained with caption cross-entropy. Two variants: an MLP mapper with GPT-2 fine-tuned (156M params), or a transformer mapper with GPT-2 frozen (43M params). Author-reported: Conceptual Captions CIDEr 87.26 (MLP variant) vs 77.57 for VLP; nocaps overall CIDEr 65.83 (transformer) vs 63.8 for Oscar, which uses tags.Paper · 2021-11-18DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding SharingReplaces DeBERTa's masked language modeling with ELECTRA-style replaced token detection and shares embeddings between generator and discriminator with stop-gradient on the generator side (GDES), avoiding the conflicting "tug-of-war" updates. DeBERTaV3-Large reaches 91.37 GLUE average (author-reported, dev set).Paper · 2021-11-18Enhanced Membership Inference Attacks against Machine Learning ModelsFrames membership inference as a hypothesis test and builds four black-box attacks whose loss thresholds use more of the target: S (shadow models), P (population records), R (per-record reference models) and D (self-distilled models). Thresholds fix the false-positive rate. On Purchase100 setup II, AUC rises from 0.754 (S) to 0.831 (D), author-reported.Paper · 2021-11-18Merging Models with Fisher-Weighted AveragingFisher merging averages the parameters of models that share an initialization, weighting each parameter by its diagonal Fisher information instead of taking a plain mean. It beats plain averaging for BERT ensembles and WiSE-FT fine-tuning. Merging with a donor model is competitive with gradient-based intermediate-task training and, by the authors' FLOP estimate, about 6x cheaper. Author-reported.Paper · 2021-11-18SimMIM: A Simple Framework for Masked Image ModelingSimMIM pre-trains vision Transformers by replacing randomly chosen 32x32 patches with a learnable mask token and regressing the raw pixels of the masked area with a single linear layer and an l1 loss. Author-reported: 83.8% ImageNet-1K fine-tuning top-1 with ViT-B (+0.6 over BEiT) and 87.1% with SwinV2-H at 512^2, using ImageNet-1K only.Paper · 2021-11-18Swin Transformer V2: Scaling Up Capacity and ResolutionScales the Swin vision transformer to 3 billion parameters and 1,536-pixel inputs. Residual post-norm with scaled cosine attention keeps training stable; a log-spaced continuous position bias transfers attention biases across window sizes; SimMIM pretraining cuts the labelled data needed. Author-reported: 84.0% ImageNet-V2 top-1, 63.1 COCO test-dev box AP, 59.9 ADE20K mIoU.Paper · 2021-11-17ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D DataARKitScenes: over 5,000 RGB-D captures of 1,661 indoor scenes recorded in homes with an iPad Pro LiDAR, plus laser-scanner depth and oriented boxes for 17 furniture classes. Two benchmarks: 3D detection (VoteNet, H3DNet, MLCVNet; whole-scene mAP 0.358, 0.383, 0.419) and colour-guided depth upsampling (MSPF best, L1 0.0149 at x4). Author-reported.Paper · 2021-11-17XLS-R: Self-supervised Cross-lingual Speech Representation Learning at ScaleXLS-R pretrains wav2vec 2.0 on about 436K hours of unlabelled speech in 128 languages, with models of 0.3B, 1B and 2B parameters, then fine-tunes for speech translation, recognition and language ID. Headline: 22.1 vs 14.7 average BLEU on CoVoST-2 X-to-English over 21 directions, beating the prior best (author-reported).Paper · 2021-11-15iBOT: Image BERT Pre-Training with Online TokenizeriBOT pre-trains Vision Transformers by masked image modeling with an online tokenizer: an EMA teacher sees unmasked views, and its token distributions supervise the masked patches, trained jointly with a DINO-style [CLS] loss. Author-reported: 82.3% linear probing (ViT-L/16, ImageNet-22K pre-training) and 84.0% fine-tuning (ViT-B/16, ImageNet-1K).Paper · 2021-11-11Masked Autoencoders Are Scalable Vision LearnersMAE pre-trains a ViT without labels: it hides 75% of image patches at random, encodes only the visible ones, and a small decoder with mask tokens reconstructs the hidden pixels. Author-reported: ViT-Huge fine-tuned at 448 px reaches 87.8% ImageNet-1K top-1 using IN1K data only, and dropping mask tokens from the encoder speeds up pre-training 3x or more.Paper · 2021-11-08Federated Learning Based on Dynamic RegularizationFedDyn: each active device minimises its local loss plus a per-round dynamic penalty (a linear term from its stored gradient and a quadratic pull toward the server model), so device optima align with global stationary points. The server averages active models and corrects them with a running state h. Author-reported: up to 9.4x fewer models sent than SCAFFOLD on CIFAR-10.Paper · 2021-11-06Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language ModelingTip-Adapter sets a CLIP-Adapter-style residual adapter's weights without gradient training. Its layers hold CLIP features of the few-shot training images and their one-hot labels, and the cache output is added to zero-shot CLIP logits. Author-reported 16-shot ImageNet accuracy (RN50): 62.03% training-free vs 60.33% zero-shot CLIP and 63.59% CLIP-Adapter; 20 epochs of key tuning give 65.51%.Paper · 2021-11-04Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language ModelsA robustness benchmark for five GLUE tasks (SST-2, QQP, MNLI, QNLI, RTE). Its adversarial examples come from 14 attacks and are filtered for cross-model transfer, fidelity and five-annotator agreement. Author-reported: ten tested models average 30.3 to 60.9 on AdvGLUE, against 85.7 to 93.2 on GLUE.Paper · 2021-11-04WaveFake: A Data Set to Facilitate Audio Deepfake DetectionWaveFake is a dataset of generated speech: about 118k clips from ten sample sets of six audio generators that re-synthesise LJSpeech and JSUT recordings, plus a TTS pipeline. Baselines are per-generator LFCC/MFCC Gaussian mixture classifiers scored by log-likelihood ratio, and a RawNet2 model. Best single-source LFCC GMM aEER is 0.062 (author-reported); out-of-distribution results vary widely.Paper · 2021-11-03An Empirical Study of Training End-to-End Vision-and-Language TransformersMETER is a fully transformer vision-language model trained end to end. A ViT or Swin image encoder and a BERT or RoBERTa text encoder are joined by six co-attention layers, pre-trained with masked language modeling and image-text matching on about 4M pairs. Author-reported VQAv2 test-std accuracy is 77.64, above region-feature VinVL's 76.60. A 14M-image version reaches 80.54.Paper · 2021-11-03LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text PairsReleases LAION-400M, about 400 million image-text pairs taken from Common Crawl alt-text and filtered with CLIP (image-text cosine similarity at least 0.3), plus CLIP embeddings and kNN indices. A DALL-E replication trained on a 7.2M-pair subset is shown only as samples; no quantitative benchmark is reported.Paper · 2021-11-01Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A SurveySurvey of NLP methods built on large pre-trained transformer language models (BERT, GPT, T5). It organizes the work into three paradigms: pre-train then fine-tune, prompt-based learning, and NLP as text generation. It also covers generating training data with these models and open questions. The survey runs no experiments of its own.Paper · 2021-10-29CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation MethodPINNs whose PDE loss uses only automatic differentiation need dense collocation points. CAN-PINN couples AD slopes with numerical-differentiation stencils over neighbouring points (upwind for convection, central for pressure), so sparse samples still constrain the flow. Author-reported: it beats AD-only and stencil-only PINNs on several flow problems.Paper · 2021-10-27Adversarial Neuron Pruning Purifies Backdoored Deep ModelsAdversarial Neuron Pruning (ANP) learns a mask per neuron (per BatchNorm channel in the code) while weights and biases are adversarially perturbed, then prunes low-mask neurons without fine-tuning. With 1% of CIFAR-10 clean data, ResNet-18 ASR falls below 1% on five of six attacks (CLB 3.98%), with a 1.47-point mean clean-accuracy drop. Author-reported.Paper · 2021-10-27Towards artificial general intelligence via a multimodal foundation modelPre-trains BriVL, a two-tower image-text model with momentum encoders and negative queues (MoCo-style InfoNCE), on 650M weakly correlated Chinese web pairs, with no object detector. Author-reported: 82.41% zero-shot on UCM at a 5/16 unseen/seen split, against 76.02% for CLIP ResNet-50x4; 62.38% zero-shot on Toutiao News, against 42.55% for RoBERTa-large.Paper · 2021-10-26Conflict-Averse Gradient Descent for Multi-task LearningCAGrad merges the task gradients into one update. It starts from their average and picks the direction that most improves the worst task, staying within a radius c of the average. c=0 gives plain gradient descent. For c<1 it provably converges to a minimum of the average loss. Author-reported: best NYU-v2 and CityScapes Δm, best Meta-World MT10 and MT50 success without metadata.Paper · 2021-10-21Fast Model Editing at ScaleMEND trains small editor networks that turn a layer's fine-tuning gradient, kept as a rank-1 outer product of layer input and output gradient, into a low-rank weight edit. One desired input-output pair then edits a pre-trained model in one step. Author-reported: on GPT-J (6B) edit success 0.88 with perplexity drawdown 0.031; on T5-XXL (11B) 0.89 with drawdown under 0.001.Paper · 2021-10-20Human-Centered Explainable AI (XAI): From Algorithms to User ExperiencesSurvey chapter, not a method: maps XAI techniques to the user questions they answer, reviews HCI findings on XAI pitfalls such as unwarranted trust and weak task gains, and proposes theory-driven, human-compatible XAI. Contains no experiments and no headline numbers.Paper · 2021-10-18Understanding Dimensional Collapse in Contrastive Self-supervised LearningShows that contrastive self-supervised learning also suffers dimensional collapse, where embeddings span a low-dimensional subspace. Traces it to strong augmentation and to implicit low-rank bias in deep linear nets. Proposes DirectCLR, which drops the projector and applies InfoNCE to a fixed sub-vector of the encoder output. Reports 62.7% vs 61.1% for SimCLR with a linear projector on ImageNet.Paper · 2021-10-15BBQ: A Hand-Built Bias Benchmark for Question AnsweringBBQ is a hand-built QA benchmark of 58,492 examples in nine social-bias categories plus two intersectional ones. Each template pairs an ambiguous and a disambiguated context with a negative and a non-negative question. Its best model, UnifiedQA-11B in RACE format, reaches 77.8% accuracy, and 76-77% of its wrong answers in ambiguous contexts align with bias (author-reported).Paper · 2021-10-15FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo LabelingFlexMatch adds Curriculum Pseudo Labeling to FixMatch. Each class gets its own confidence threshold, scaled by how many unlabeled samples are currently assigned to it, so classes the model is still learning admit more pseudo labels. It adds no parameters and no forward passes. Author-reported: 39.94% error on CIFAR-100 with 400 labels, against 46.42% for FixMatch.Paper · 2021-10-15Graph Neural Networks with Learnable Structural and Positional RepresentationsAdds a separate, learnable positional vector to each node of message-passing and Transformer GNNs (LSPE). Positions start from random-walk return probabilities (RWPE) and are updated at every layer; an optional Laplacian loss regularises them. Author-reported: GatedGCN-LSPE reaches test MAE 0.090 on ZINC versus 0.251 without positional encoding.Paper · 2021-10-15Multitask Prompted Training Enables Zero-Shot Task GeneralizationFine-tunes T5 LM-adapted encoder-decoders (3B and 11B) on 62 NLP datasets, each written as several natural-language prompts, then tests zero-shot on held-out NLI, coreference, sentence completion and WSD. T0 matches or beats GPT-3 on 9 of 11 held-out datasets (author-reported).Paper · 2021-10-14P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and TasksP-tuning v2 adds trainable prompt vectors to every layer of a frozen pretrained model, plus a trainable classification head. Author-reported: comparable to fine-tuning on SuperGLUE dev from BERT-large to GLM-10B with 0.1-3% trainable parameters, and competitive on sequence labeling and extractive QA, where earlier prompt tuning lagged far behind.Paper · 2021-10-13DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D QueriesDETR3D predicts 3D boxes from six cameras with no depth network and no NMS. 900 learned queries each hold a 3D reference point, projected into every image to sample features; six decoder layers refine them under a Hungarian set loss. Reports 0.374 NDS / 0.303 mAP on nuScenes val, 0.434 NDS with CBGS (author-reported).Paper · 2021-10-12Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective AnnotationsTrains one BERT encoder with a separate output head per annotator (multi-task), so each annotator's labels are learned directly and the majority vote is taken over the heads at prediction time. Author-reported: on seven binary tasks it matches or beats majority-label training, and its per-annotator spread correlates better with annotator disagreement than softmax or MC dropout.Paper · 2021-10-12Open-Set Recognition: a Good Closed-Set Classifier is All You Need?Open-set recognition (rejecting inputs from unseen classes) tracks closed-set accuracy, with Pearson 0.95 on standard benchmarks. Stronger training (600 epochs, cosine restarts, RandAugment, label smoothing on TinyImageNet) plus max-logit scoring lifts the plain softmax baseline to match or beat prior methods. A new fine-grained Semantic Shift Benchmark shows the baseline about level with ARPL+.Paper · 2021-10-11Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D ScansOmnidata renders steerable multi-task vision datasets from textured 3D scans: sampled cameras and points, then 21 mid-level cues per view. Its 14.6M-image starter set trains a DPT-Hybrid depth model that beats MiDaS v3 zero-shot on NYU and OASIS, and a UNet for surface normals that reaches at or above approximate human AUC_p on OASIS (author-reported).Paper · 2021-10-07Darts: User-Friendly Modern Machine Learning for Time SeriesDarts is a Python time-series library with one fit()/predict() API over classical, deep and probabilistic forecasters, a TimeSeries container for multivariate and sampled series, and past and future covariates. The paper reports no accuracy benchmarks; its example trains a TCN with a Laplace likelihood on two series.Paper · 2021-10-06CLIP-Forge: Towards Zero-Shot Text-to-Shape GenerationGenerates 3D shapes from text without any text-shape pairs. A voxel autoencoder learns 128-d shape embeddings; a conditional RealNVP flow maps frozen CLIP features of rendered views to those embeddings, and CLIP text features replace the image features at test time. Author-reported: FID 2425.25 and 83.33% category accuracy on 224 prompts, ahead of two text-supervised baselines.Paper · 2021-10-05Data Augmentation Approaches in Natural Language Processing: A SurveySurvey of data augmentation for NLP. Groups methods by how diverse the augmented data is: paraphrasing (small rewordings), noising (added swaps, deletions, insertions, substitutions) and sampling (new text from rules, generators, self-training or mixup). Covers tricks for stacking, tuning and filtering, and which tasks use which kind. No experiments, so no benchmark numbers.Paper · 2021-10-04Trustworthy AI: From Principles to PracticesSurvey and practitioner guide. Defines eight aspects of AI trustworthiness (robustness, generalization, explainability, transparency, reproducibility, fairness, privacy, accountability), maps existing approaches onto the AI lifecycle, and proposes TrustAIOps, a continuous MLOps-style workflow. No new method or experiments in the main text; no numeric results.Paper · 2021-10-02FairFed: Enabling Group Fairness in Federated LearningServer-side aggregation rule for federated learning. It reweights client models by how closely each client's local equal-opportunity gap matches the global gap, so fairness can improve without sharing per-group statistics, with any local debiasing method. Author-reported: at alpha=0.1, Adult EOD goes from -0.174 (FedAvg) to -0.017 (FairFed with reweighting), accuracy from 0.835 to 0.830.Paper · 2021-10-02Optimizing Neural Network for Computer Vision task in Edge DeviceTrains a small two-convolution CNN to recognise five hand gestures from camera images, then reduces the trained model's precision with TensorFlow Lite so it fits and runs on a Raspberry Pi. Reports 98.1% validation accuracy at 96x96 before conversion and 96.9% on a 32-image test set after, with the model size falling to 15.9 MB.Paper · 2021-10-01A survey on datasets for fairness-aware machine learningSurvey of 15 tabular datasets used in fairness-aware ML, in finance, criminology, healthcare, society and education. For each it learns a Bayesian network to show links to protected attributes and class, then scores logistic regression on a 70/30 split with statistical parity, equalized odds and ABROCA. Accuracy ranges from 0.63 to 1.00; several zero fairness gaps come from constant predictions.Paper · 2021-09-30MatSciBERT: A Materials Domain Language Model for Text Mining and Information ExtractionMatSciBERT continues masked-language-model pretraining from SciBERT on about 285M words of materials-science full text and abstracts, then fine-tunes it for NER, abstract classification and relation classification. Author-reported: it beats SciBERT on every reported task, with SOFC-Slot test Macro-F1 up about 4.2 points for the BiLSTM-CRF head.Paper · 2021-09-30PubTables-1M: Towards comprehensive table extraction from unstructured documentsA dataset of nearly one million PubMed Central tables, with cell, row, column and header boxes. PDF text is aligned to XML markup and oversegmented headers are merged (canonicalization). DETR trained on it handles detection, structure recognition and functional analysis. Author-reported: 0.8138 content accuracy on all test tables for DETR, against 0.1039 for Faster R-CNN.Paper · 2021-09-29Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue SystemPPTOD is one T5 model for task-oriented dialogue. Each sub-task (intent, belief state, dialogue action, response) is a text-to-text prompt, and the model is pre-trained on eleven partially annotated dialogue corpora. On MultiWOZ 2.0 end-to-end it reports a combined score of 102.92 (PPTOD-base), the best of the systems it compares against, and it is strongest in low-resource DST.Paper · 2021-09-28Multiwavelet-based Operator Learning for Differential EquationsMWT is a neural operator that maps input functions to PDE solutions. Fixed Legendre or Chebyshev multiwavelet filters split signals into scales, and small learned kernels act per scale, reused at every scale. Author-reported: the Legendre variant has the lowest relative L2 error of the listed baselines on KdV, Burgers, Darcy and Navier-Stokes.Paper · 2021-09-27Training Spiking Neural Networks Using Lessons From Deep LearningTutorial and perspective on training spiking neural networks with deep-learning tools. Derives the discrete leaky integrate-and-fire neuron, compares input and output spike codes, loss functions and credit-assignment rules, and explains surrogate-gradient backpropagation through time and its link to STDP. Presents no new method and no experiments of its own.Paper · 2021-09-25FedProc: Prototypical Contrastive Federated Learning on Non-IID dataFedProc is a FedAvg-based federated learning method for non-IID image classification. Each client trains with a prototypical contrastive loss that pulls its features toward the server's averaged class prototypes, plus cross-entropy, weighted by a per-round schedule. Author-reported: 70.7% on CIFAR-10, 74.6% on CIFAR-100 and 35.4% on Tiny-ImageNet, 1.6 to 7.9 points above MOON.Paper · 2021-09-20Description of Corner Cases in Automated Driving: Goals and ChallengesExtended abstract (position paper, no experiments) arguing that automated-driving corner cases need a machine-interpretable description that combines existing scenario description languages with corner-case levels. It gives six research questions and three conceptual use cases (glare, unknown objects, occlusion). It reports no quantitative results and no implementation.Paper · 2021-09-16OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle CommunicationOPV2V is a simulated open dataset for vehicle-to-vehicle perception and a 16-model LiDAR fusion benchmark. It proposes Attentive Intermediate Fusion: compressed feature sharing, then self-attention across connected vehicles. Author-reported: VoxelNet reaches 0.864 AP@0.7 on default towns with intermediate fusion vs 0.526 without; about 3% AP loss at 4096x compression.Paper · 2021-09-13CEM: Commonsense-aware Empathetic Response GenerationCEM writes empathetic replies using commonsense. COMET infers five relations about the user's last turn (xReact, xWant, xNeed, xIntent, xEffect); affective and cognitive encoders fuse them into a Transformer decoder, trained with NLL, emotion cross-entropy and a diversity loss. On EmpatheticDialogues: PPL 36.11 (best baseline 36.93), Dist-1 0.66, Dist-2 2.99 (author-reported).Paper · 2021-09-10R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping packageR3LIVE fuses LiDAR, IMU and camera. A LiDAR-inertial odometry builds a global point map; a visual-inertial odometry colours each map point by minimising the photometric error of tracked points, without visual features. Author-reported: 0.093-0.164 m translation and 0.285-3.925 deg rotation drift over four 1.2-1.5 km campus runs.Paper · 2021-09-09PPT: Pre-trained Prompt Tuning for Few-shot LearningPrompt tuning trains only soft prompt vectors on a frozen model, but with few labels it does poorly from random starts. PPT pre-trains the prompts on unlabeled text with three self-supervised cloze tasks, then tunes them per task. With 11B models and 32 examples per task, the authors report results matching or beating full fine-tuning on several benchmarks.Paper · 2021-09-07nnFormer: Interleaved Transformer for Volumetric SegmentationnnFormer is a 3D encoder-decoder transformer for volumetric medical segmentation. It alternates convolutional embedding and down-sampling with local-volume and global self-attention, and replaces U-Net skip connections with skip attention. Author-reported: lower HD95 than nnUNet on three datasets and about equal DSC; on Synapse nnUNet has the higher average DSC.Paper · 2021-09-06Voxel Transformer for 3D Object DetectionVoTr replaces the 3D sparse-convolution backbone of voxel detectors with Transformer blocks. Sparse and submanifold voxel modules attend over non-empty voxels with local and dilated ranges (48 voxels per query) and a GPU hash table. Author-reported: Waymo LEVEL_1 3D mAP 74.95 (VoTr-TSD) and KITTI test moderate car AP 82.09.Paper · 2021-09-02Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and BeyondSurvey of causal inference in NLP with two parts: estimating causal effects when text is a confounder, outcome or treatment, and using causal formalisms for robust, fair and interpretable prediction. Gives the identification assumptions (ignorability, positivity, consistency) and the counterfactual view of spurious correlations. Reports no experiments; open problems are listed per section.Paper · 2021-09-01Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment AnalysisMultiModal InfoMax (MMIM) trains a sentiment regressor whose fusion vector keeps mutual information with the text, visual and acoustic inputs. It uses a Barber-Agakov bound with a Gaussian-mixture entropy term on the input pairs and a CPC-style contrastive loss on the fusion. Author-reported CMU-MOSEI: MAE 0.526, Corr 0.772, Acc-7 54.24.Paper · 2021-08-30Deep Reinforcement Learning at the Edge of the Statistical PrecipiceDeep RL results from 3 to 10 runs carry large statistical uncertainty. Proposes stratified bootstrap CIs, score-distribution performance profiles, interquartile mean, optimality gap and probability of improvement. On Atari 100k, sample medians vary and are biased with few runs. Author-reported; released as the rliable library.Paper · 2021-08-26Federated Reinforcement Learning: Techniques, Applications, and Open ChallengesSurvey of federated reinforcement learning (FRL), where RL agents share learned models or gradients instead of raw experience. Splits FRL into horizontal (similar tasks, separate environments) and vertical (one shared environment, partial observations). Reviews applications in edge computing, communication, control and attack detection, and lists open issues. Has no experiments of its own.Paper · 2021-08-26Machine Unlearning of Features and LabelsRemoves sensitive feature values or wrong labels from a trained model with a closed-form influence-function update instead of retraining. First-order needs a tuned rate; second-order has a certified-unlearning bound for strongly convex losses. Author-reported: 90x speed-up for first-order on one linear task; an LSTM canary drops to near-zero exposure.Paper · 2021-08-25Transformer for Single Image Super-ResolutionESRT is a lightweight super-resolution network: a CNN backbone that processes features at half resolution while keeping a high-frequency residual, then one efficient Transformer whose attention runs over unfolded 3×3 patches split into 4 segments. Author-reported: ~0.75M params, best or near-best PSNR among lightweight ×3/×4 models, 4,191M vs 16,057M training memory for a plain Transformer.Paper · 2021-08-25YOLOP: You Only Look Once for Panoptic Driving PerceptionOne shared encoder (CSPDarknet with SPP and FPN) feeds three heads that detect vehicles, segment the drivable area and segment lane lines on BDD100K, trained end to end in one forward pass. Author-reported: 41 FPS on a TITAN XP, 23 FPS on a Jetson TX2, vehicle detection mAP50 76.5, drivable-area mIoU 91.5, lane accuracy 70.50 and lane IoU 26.20.Paper · 2021-08-24Isaac Gym: High Performance GPU-Based Physics Simulation For Robot LearningIsaac Gym runs PhysX physics, observations, rewards and PPO training on one GPU, passing physics buffers to PyTorch as tensors without CPU copies. Author-reported on one A100: Ant reaches reward 3000 in about 20 s, Humanoid reaches 5000 in under 4 min, and the OpenAI-style Shadow Hand task reaches 20 consecutive successes in under 1 h, versus about 30 h for OpenAI's CPU-cluster setup.Paper · 2021-08-20DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic PredictionSurveys deep models for urban traffic prediction (grid-based, graph-based and multivariate time-series) and benchmarks grid and graph models under one protocol on TaxiBJ, BikeNYC, TaxiNYC, METR-LA, PeMS-BAY and PeMSD7(M). Author-reported: STDN has the best grid RMSE on all four grid datasets; Graph WaveNet is best at 60 minutes on METR-LA and MTGNN on PeMS-BAY.Paper · 2021-08-16On the Opportunities and Risks of Foundation ModelsA 200-page report that defines foundation models as broadly trained, adaptable models and surveys their capabilities, applications, technical building blocks and societal risks. It has no experiments of its own. Main claims: emergence and homogenization drive both gains and risks; adaptation choices depend on compute, task data and gradient access.Paper · 2021-08-13Conditional DETR for Fast Training ConvergenceConditional DETR speeds up DETR training by splitting decoder cross-attention into a content part and a spatial part, with the spatial query predicted per object from a reference point and the previous decoder embedding. Author-reported COCO val: 6.7x faster convergence for R50/R101 and 10x for DC5 backbones; R50 reaches 40.9 AP in 50 epochs vs 34.9 for DETR.Paper · 2021-08-11Rethinking Coarse-to-Fine Approach in Single Image DeblurringMIMO-UNet is a single U-Net for single-image deblurring. It takes the blurry image at three scales, its one decoder outputs a sharp estimate at each scale, and asymmetric feature fusion merges features across levels. Trained with a multi-scale L1 loss plus a 0.1-weighted frequency loss. Author-reported on GoPro: 31.73 dB PSNR at 0.008 s for MIMO-UNet, 32.68 dB for MIMO-UNet++ (with self-ensemble).Paper · 2021-08-10Post-hoc Interpretability for Neural NLP: A SurveySurvey, not a method. Sorts post-hoc interpretability methods for neural NLP by how they communicate explanations: input features, adversarial and counterfactual examples, influential training examples, natural-language rationales, concepts, vocabulary projections, ensembles, probing, and rules. Gives each family's mechanics, its validation, and open problems. No new experiments.Paper · 2021-08-09Noisy Channel Language Model Prompting for Few-Shot Text ClassificationScores each label by how well it explains the input, P(x given label), instead of P(label given x), using a frozen GPT-2 Large with demonstrations, ensembled demonstrations or prompt tuning. Author-reported on 11 text classification datasets with 16 examples: channel prompt tuning averages 61.7% (worst case 53.0%), against 57.7% (47.1%) for direct head tuning.Paper · 2021-08-03Robust Compressed Sensing MRI with Deep Generative PriorsPosterior sampling for compressed-sensing MRI: a score-based prior trained on T2 fastMRI brain slices, with no knowledge of the sampling pattern, reconstructs multi-coil k-space by annealed Langevin dynamics. Author-reported: competitive with supervised baselines in-distribution and more robust under mask and anatomy shifts, at about 16 minutes per high-resolution brain on an RTX 2080Ti.Paper · 2021-08-02Evaluating Federated Learning for Intrusion Detection in Internet of Things: Review and ChallengesMultinomial logistic regression detects attacks in CIC-ToN-IoT, split over 10 clients in three non-IID settings (basic, balanced, mixed) and aggregated with FedAvg or Fed+. Fed+ beats FedAvg in all three federated settings. Author-reported final mean client accuracy in the mixed setting: 0.8869 with Fed+, 0.8423 with FedAvg.Paper · 2021-08-02Shallow Attention Network for Polyp SegmentationSANet segments colon polyps with a Res2Net encoder and a decoder that gates shallow features with deep ones. Three additions: a colour-exchange augmentation in LAB space, a shallow attention module, and a logit correction applied at inference only. Reports the best mean Dice and IoU on five polyp benchmarks, at about 72 FPS on an RTX 2080Ti (author-reported).Paper · 2021-08-02StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image GeneratorsShifts a pretrained StyleGAN2 to a new domain from text prompts alone, with no training images. A frozen copy supplies source images, while the trained copy is pushed so its CLIP-space change matches the text direction; for larger shape changes, only the k layers a short latent optimisation changes most are updated. Author-reported: judged higher quality than most 5- and 10-shot baselines.Paper · 2021-07-31Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural NetworksGRIL, a bidirectional recurrent graph network for multivariate time series imputation. Each direction runs a message-passing GRU encoder and a spatial decoder that fills gaps from neighbouring sensors in two stages per step. Author-reported MAE is 13% to 55% below BRITS on the real-data settings (AQI out-of-sample: 27%).Paper · 2021-07-29Rethinking and Improving Relative Position Encoding for Vision TransformeriRPE adds 2D relative position encoding to vision transformer attention: a learned value per bucket of each patch offset, log-spaced beyond a threshold and direction-aware (Product), on keys, queries or values. DeiT-S reaches 81.4% top-1 with all three (80.9 keys only; baseline 79.9). DETR gains 1.3 AP at 150 epochs and 1.7 at 300 (author-reported).Paper · 2021-07-29Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-SeriesSTraTS models a sparse, irregularly sampled clinical time series as (time, variable, value) triplets, with learned continuous-value embeddings, a Transformer encoder and attention pooling. Forecasting the next 2 hours serves as self-supervised pretraining. Author-reported: best ROC-AUC, PR-AUC and min(Re,Pr) on MIMIC-III and PhysioNet-2012 mortality, PR-AUC up 3.2% and 3.5%.Paper · 2021-07-28Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingSurvey that defines prompt-based learning: a pre-trained language model fills a template-built prompt, so tasks are cast as text filling. It gives a formal notation (prompt, answer search, answer mapping), a typology of templates, answers, multi-prompt methods and tuning strategies, plus applications and open challenges. The paper has no experiments of its own.Paper · 2021-07-27CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing FlowsDetects and localizes anomalies from anomaly-free images only. A frozen ImageNet encoder gives features at three scales, and one small conditional normalizing flow per scale scores each location, conditioned on a 2D sin/cos position code. Author-reported on MVTec AD: 98.26 image AUROC, 98.62 pixel AUROC and 94.60 AUPRO, with claimed 10x faster and smaller models than prior state of the art.Paper · 2021-07-27Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series SignalsCAE-M flags anomalies in multi-sensor time series with a convolutional autoencoder whose latent codes get an MMD penalty toward a Gaussian, trained jointly with an attention BiLSTM and an autoregressive predictor. Author-reported mean F1: 0.964 (PAMAP2), 0.996 (CAP), 0.996 (fatigue), above all ten baselines in its table.Paper · 2021-07-26Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action RecognitionCTR-GC gives each channel of a skeleton graph convolution its own topology: a learned shared adjacency matrix refined by per-sample, per-channel joint correlations. Stacked in CTR-GCN with multi-scale temporal convolutions. Author-reported: 88.9% X-Sub and 90.6% X-Set on NTU RGB+D 120, 92.4% X-Sub on NTU RGB+D, 96.5% top-1 on NW-UCLA.Paper · 2021-07-26Continental-Scale Building Detection from High Resolution Satellite ImageryU-Net building segmentation on 50 cm African satellite imagery, with a ResNet-50-v2 encoder, residual decoder blocks, Gaussian edge-weighted cross-entropy plus focal Tversky loss, mixup, ImageNet pre-training and soft-label self-training. Connected components turn pixel masks into instances. Author-reported: self-training adds mAP +0.057 on average; the pipeline produced 516M footprints.Paper · 2021-07-23Machine Learning with a Reject Option: A surveySurvey of machine learning with a reject option. Defines ambiguity and novelty rejection, reviews metrics for models with rejection (fixed rate, accuracy-reject curves, cost functions), and groups methods into separated, dependent and integrated rejectors, plus rejector combinations. Runs no experiments; reports no new results.Paper · 2021-07-21Reading Race: AI Recognises Patient's Racial Identity In Medical ImagesTrains standard CNNs (ResNet34, DenseNet121, EfficientNetB0) to predict self-reported race from nine medical imaging datasets, then probes why. Author-reported one-vs-all ROC-AUC is 0.96-0.99 on chest X-rays, with external validation. Race stays above chance after noise, blur, frequency filtering, masking or lung removal, so no single cause is found.Paper · 2021-07-16Align before Fuse: Vision and Language Representation Learning with Momentum DistillationALBEF aligns image and text features with a contrastive loss before fusing them with cross-modal attention, and adds momentum distillation: an EMA teacher supplies soft targets to learn from noisy web pairs. Pre-trained on 4M or 14M images, it reports state-of-the-art results on several vision-language tasks, e.g. VQA test-std 76.04 and NLVR2 test-P 83.14 (14M, author-reported).Paper · 2021-07-15Partially Observable Markov Decision Processes (POMDPs) and RoboticsReview of POMDP planning for robotics. Exact solving is intractable, so the paper surveys sampling-based approximate solvers (PBVI, Perseus, HSVI2, SARSOP, POMCP, ABT, DESPOT and others), grouped by the five difficulties they address, plus software and tips. Headline: SARSOP reaches a better Tag policy than HSVI2 in 6 seconds, against about 50 hours for PBVI (cited, author-reported).Paper · 2021-07-13HDMapNet: An Online HD Map Construction and Evaluation FrameworkHDMapNet predicts vectorized local HD map elements (lane dividers, boundaries, pedestrian crossings) in bird's-eye view from cameras, LiDAR, or both. Per-camera MLPs and extrinsics lift camera features to BEV, pillars encode LiDAR, and three heads predict segmentation, embeddings and directions. Author-reported on nuScenes: camera-LiDAR fusion reaches 44.5 IoU and 30.6 mAP, the best compared.Paper · 2021-07-13OpenCDA: An Open Cooperative Driving Automation Framework Integrated with Co-SimulationOpen Python framework for cooperative driving automation: couples CARLA vehicle simulation with SUMO traffic, a full-stack driving stack and a scenario manager. Demonstrated on platooning: author-reported followers' time gap near 0.6 s, zero hazard events, and a fuzzy-logic merge (GFS) that keeps a larger gap than a nearest-member heuristic.Paper · 2021-07-07A Survey of Uncertainty in Deep Neural NetworksSurvey of uncertainty estimation in deep neural networks. Separates data (aleatoric), model (epistemic) and distributional uncertainty, then reviews deterministic, Bayesian, ensemble and test-time-augmentation methods, uncertainty measures and calibration metrics. Runs no experiments of its own; its headline claims are quoted from cited works.Paper · 2021-07-07Disentangle Your Dense Object DetectorDDOD adds three decoupling steps to one-stage detectors: separate cost-based label assignment for classification and regression, a deformable first conv in each head branch, and per-FPN-level loss weights from positive counts. Author-reported: +2.2 mAP on ATSS (39.4 to 41.6, COCO val, ResNet-50, 1x); 55.0 mAP on test-dev with extra test-time methods.Paper · 2021-07-06Depth-supervised NeRF: Fewer Views and Faster Training for FreeDS-NeRF adds a depth term to NeRF training. Sparse 3D keypoints that COLMAP produces while estimating camera poses give each ray a depth target, and the loss pulls the ray-termination distribution toward a Gaussian around it. On NeRF Real with 2 views it reports 20.2 PSNR versus 13.5 for NeRF, and 2-3x fewer iterations to reach NeRF's peak PSNR (author-reported).Paper · 2021-07-01Variational Diffusion ModelsGaussian diffusion model with a learned monotone noise schedule, Fourier features and a U-Net noise predictor, trained on the continuous-time variational bound (a schedule-weighted noise-prediction MSE). Author-reported test bits per dim: 2.65 CIFAR-10, 2.49 with augmentation, 3.72 ImageNet 32, 3.40 ImageNet 64, the best of the compared models.Paper · 2021-06-30A Diverse Corpus for Evaluating and Developing English Math Word Problem SolversIntroduces ASDiv, 2,305 English math word problems labelled with problem type and grade level, and a BLEU-based lexicon usage diversity (LD, averaged into CLD) to measure how varied a corpus is. One solver reaches 0.86 on a low-diversity MathQA subset, while three solvers reach about 0.36 on ASDiv (author-reported).Paper · 2021-06-30All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated TextCrowd raters on Amazon Mechanical Turk judged whether stories, news articles and recipes were human- or GPT-3-written. Untrained raters were at chance on GPT-3 text (49.9%). Three quick trainings raised accuracy; only labelled examples gave a significant gain (55% overall). Author-reported; the authors advise against small untrained human evaluations of current models.Paper · 2021-06-28Multimodal Trajectory Prediction Conditioned on Lane-Graph TraversalsPGP predicts several future paths of a target vehicle. A GRU and graph encoder over lane-graph nodes feeds a behaviour-cloned policy that samples routes; an attention decoder conditioned on each route and a 5-d Gaussian latent outputs trajectories, clustered to K=10. Author-reported on nuScenes: MinADE10 1.00, MissRate10,2 0.37, offroad 0.03; best or tied on four of five metrics.Paper · 2021-06-26Time-Series Representation Learning via Temporal and Contextual ContrastingTS-TCC pretrains a time-series encoder without labels, using two augmented views, a Transformer that predicts future latent steps across the views, and a contrastive loss on the resulting contexts. A linear classifier on the frozen encoder reaches 90.37% accuracy on HAR, 83.00% on Sleep-EDF and 97.23% on Epilepsy (author-reported).Paper · 2021-06-25PVT v2: Improved Baselines with Pyramid Vision TransformerPVT v2 upgrades the Pyramid Vision Transformer backbone with three changes: attention whose keys and values are average-pooled to a fixed 7x7 grid (linear cost), overlapping patch embedding, and a 3x3 depthwise conv inside the feed-forward block. B5 reaches 83.8% ImageNet top-1 and B4 reaches 46.1 RetinaNet AP on COCO (author-reported).Paper · 2021-06-24Spatial-Temporal Graph ODE Networks for Traffic Flow ForecastingSTGODE forecasts traffic flow by writing graph propagation over a spatial and a DTW-based semantic road graph as a tensor ODE with a restart term, plus dilated temporal convolutions, so deep models keep distant information. Author-reported: lowest RMSE, MAE and MAPE on six PeMS datasets, one hour in and 60 minutes out.Paper · 2021-06-24Video Swin TransformerApplies Swin Transformer's shifted-window attention to video: self-attention runs inside 3D windows of 8x7x7 tokens over 2x4x4 patches, with the windows shifted in time and space every other block. It starts from ImageNet Swin weights and reports 84.9% top-1 on Kinetics-400 and 69.6% on Something-Something v2 (author-reported).Paper · 2021-06-22BARTScore: Evaluating Generated Text as Text GenerationScores generated text by the average log-likelihood a pre-trained BART encoder-decoder assigns to one text given another (source, reference or hypothesis), with no human labels. Prompts and fine-tuning on CNN/DM or ParaBank2 are optional add-ons. Author-reported: it outperforms existing top-scoring metrics in 16 of 22 test settings.Paper · 2021-06-22DocFormer: End-to-End Transformer for Document UnderstandingDocFormer is an encoder-only transformer for document understanding. It fuses text tokens, a ResNet50 image grid and word-box features in a multi-modal self-attention layer with spatial weights shared across modalities, and is pre-trained on 5M IIT-CDIP pages. Base reaches 83.34 F1 on FUNSD and 96.17% on RVL-CDIP (author-reported).Paper · 2021-06-22P2T: Pyramid Pooling Transformer for Scene UnderstandingPyramid Pooling Transformer (P2T) shortens the keys and values of self-attention with average pooling at four scales, adds a depthwise convolution to each pooled map, and uses an inverted-bottleneck feed-forward with depthwise convolution. Author-reported: P2T-Small reaches 82.4 top-1 on ImageNet-1K, 46.7 mIoU on ADE20K (Semantic FPN) and 45.5 box AP on COCO (Mask R-CNN).Paper · 2021-06-213D Object Detection for Autonomous Driving: A SurveySurvey of 3D object detection for autonomous driving. Groups methods by input (image, point cloud, camera plus LiDAR fusion), compares KITTI, nuScenes and Waymo results, and re-runs 15 detectors for runtime, error and robustness. Author-reported: 3D location error is the largest error source, and BtcDet has the top KITTI test Car moderate AP3D (82.86).Paper · 2021-06-21Dive into Deep LearningOpen textbook that teaches deep learning from linear algebra and probability through CNNs, RNNs, attention and transformers, optimization and reinforcement learning. Each concept comes with runnable Jupyter code for several frameworks. Text is CC BY-SA 4.0. The book reports no benchmark results of its own.Paper · 2021-06-20NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view ReconstructionNeuS reconstructs surfaces from posed images by training a neural SDF with volume rendering. It converts the SDF into an opaque density whose rendering weights peak exactly at the zero-level set (unbiased to first order) while still handling occlusion. On 15 DTU scenes it reports mean Chamfer 0.77 with masks (IDR 0.90) and 0.84 without (UNISURF 1.02).Paper · 2021-06-18End-to-end Temporal Action Detection with TransformerTadTR detects actions in untrimmed video with a Transformer. Learnable action queries predict segments in parallel, matched one-to-one to ground truth, so no anchors or NMS are needed. It adds temporal deformable attention, iterative segment refinement and an actionness head. Author-reported: 56.7 average mAP on THUMOS14, 32.09 on HACS Segments, 36.75 on ActivityNet-1.3 with an extra classifier.Paper · 2021-06-17XCiT: Cross-Covariance Image TransformersXCiT replaces token-to-token self-attention with cross-covariance attention (XCA), which attends across feature channels. Its l2-normalised query-key channel map makes cost linear in token count. Blocks add a depth-wise local patch interaction and an FFN. Author-reported: 86.0% ImageNet-1k top-1 (L24, 8x8 patches, 384px, distilled); 48.5 COCO box AP (M24/8).Paper · 2021-06-15A White Paper on Neural Network QuantizationWhite paper on uniform affine quantization of neural networks. Post-training quantization: MSE range setting, cross-layer equalization, bias correction, AdaRound, debugging. Quantization-aware training: straight-through estimator, batch-norm folding. Author-reported: 8-bit weights and activations near float accuracy on most tested models; 4-bit weights with QAT or extra PTQ steps.Paper · 2021-06-15Dynamic Head: Unifying Object Detection Heads with AttentionsDynamic Head replaces an object detector's head with a stack of blocks that apply three attentions to a feature pyramid viewed as levels x positions x channels: scale-aware over levels, spatial-aware via deformable sampling, and task-aware per channel. Plugged into ATSS, the authors report 54.0 AP on COCO test-dev with multi-scale training and testing.Paper · 2021-06-14Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural NetworkRuns a standard ResNet-20 on CIFAR-10 images encrypted with RNS-CKKS, using bootstrapping, a minimax-composed ReLU and a homomorphic softmax. Author-reported: 98.67% agreement with plaintext ResNet-20 over 75 encrypted images, 90.67% accuracy on those images (plaintext 89.33%), and about 4 hours per image on a 112-core CPU.Paper · 2021-06-13Graph Neural Network-Based Anomaly Detection in Multivariate Time SeriesLearns a sensor graph from the cosine similarity of learned sensor embeddings, keeping the top k neighbours per sensor. A graph-attention forecaster predicts each sensor's next value. A time tick is scored by its largest sensor deviation, normalised by median and IQR and smoothed. Author-reported F1 is 0.81 on SWaT and 0.57 on WADI, above all baselines listed in the paper.Paper · 2021-06-12A Minimalist Approach to Offline Reinforcement LearningTD3+BC adds a behaviour-cloning term to the actor loss of TD3, weighted by the inverse batch mean of |Q|, and normalizes dataset states; nothing else changes. On D4RL MuJoCo v0 (15 dataset/environment pairs) it reaches a total normalized score of 979.3, against 974.6 for Fisher-BRC and 764.3 for CQL, with less than half their training time (author-reported).Paper · 2021-06-10Adversarial Graph Augmentation to Improve Graph Contrastive LearningAD-GCL trains a GIN encoder to match each graph with an edge-dropped copy, while a learned GNN augmenter gives per-edge drop probabilities and tries to make the copies differ, with a penalty on the drop ratio. Author-reported: up to 14% gain unsupervised, 6% transfer and 3% semi-supervised over GCL baselines on 18 benchmarks.Paper · 2021-06-10Pivotal Tuning for Latent-based Editing of Real ImagesPivotal Tuning Inversion (PTI) maps a real face into StyleGAN2's W latent space, then briefly fine-tunes the generator so the face is reproduced from that fixed code, with a locality term that limits side effects. Edits then run on the tuned generator. On CelebA-HQ it reports LPIPS 0.09 against 0.34 for the best baseline (author-reported).Paper · 2021-06-10Scaling Vision with Sparse Mixture of ExpertsVision MoE (V-MoE): a Vision Transformer whose every other MLP is replaced by 32 sparsely gated experts, with each patch routed to its top-2 experts. Batch Prioritized Routing drops low-priority patches to trade compute for accuracy at test time. A 14.7B model reaches 90.35% ImageNet accuracy after fine-tuning (author-reported).Paper · 2021-06-09Do Transformers Really Perform Bad for Graph Representation?Graphormer is a standard Transformer for graphs. It adds three structural encodings: learned degree embeddings on nodes, a learned attention bias per shortest-path distance, and an edge-path attention bias. It reports lower validation MAE than GIN-VN on OGB-LSC PCQM4M (0.1234 vs 0.1395) and top results on MolHIV, MolPCBA and ZINC (author-reported).Paper · 2021-06-08Flow Network based Generative Models for Non-Iterative Diverse Candidate GenerationGFlowNet trains a sequential generator so that finished objects are sampled in proportion to a positive reward, without MCMC. It learns edge flows with a log-scale flow-matching loss over a DAG of construction steps. On a molecule task with a docking-score proxy it reports top-10 reward 8.36 vs 8.06 for PPO at 10^5 samples, with fewer similar molecules.Paper · 2021-06-06Preservation of the Global Knowledge by Not-True Distillation in Federated LearningFedNTD adds a distillation term to federated learning: each client's model is trained to match the previous global model's softmax over the not-true classes. The authors link non-IID losses to forgetting. Author-reported gains over FedAvg and five other methods on MNIST, CIFAR-10/100 and CINIC-10 under sharding and LDA splits. No extra communication. Not best in every setting.Paper · 2021-06-06Tabular Data: Deep Learning is Not All You NeedBenchmarks four recent deep models for tabular data (TabNet, NODE, DNF-Net, 1D-CNN) against XGBoost on 11 datasets under one Bayesian tuning protocol. XGBoost usually wins and tunes in fewer iterations. An ensemble of the deep models with XGBoost is best (author-reported 2.32% mean relative deterioration, vs 3.34% for XGBoost).Paper · 2021-06-03Question Answering Over Temporal Knowledge GraphsAdds time to knowledge-graph question answering. Builds CronQuestions, about 410k temporal questions over a Wikidata graph with timed facts, and CronKGQA, which pairs a question encoder with TComplEx embeddings to score entities and years. Author-reported Hits@1 0.647 overall vs 0.288 for the best baseline.Paper · 2021-06-02SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-TrainingSAINT is a transformer for tabular rows that attends across the columns of each row and across the rows of a batch, with a per-column embedding. Optional pre-training combines CutMix, mixup, InfoNCE and denoising losses. Author-reported mean AUROC over 14 binary datasets is 93.13, above XGBoost (91.06) and CatBoost (90.73).Paper · 2021-06-02TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image ClassificationCorrelated multiple instance learning for whole-slide image classification: a two-layer Transformer over all patch features, with Nystrom attention and a convolutional position encoder (PPEG). Author-reported AUC: 0.9309 on CAMELYON16, 0.9603 on TCGA-NSCLC and 0.9882 on TCGA-RCC.Paper · 2021-06-01Dense Nested Attention Network for Infrared Small Target DetectionDNA-Net finds single-frame infrared small targets. Densely nested U-shaped sub-networks keep tiny targets alive in deep layers, channel-then-spatial attention re-weights each fusion, and 8-connected clustering turns the mask into target centroids. Author-reported on NUDT-SIRST (50% split): IoU 87.09 and Pd 98.73 (x10^-2), Fa 4.223 (x10^-6).Paper · 2021-05-31Emotional Voice Conversion: Theory, Databases and ESDReviews emotional voice conversion and emotional speech databases, then releases ESD: 350 parallel utterances per emotion from 10 English and 10 Chinese speakers, 5 emotions, about 29 h of audio. Reference results: speech emotion recognition 92.0% (Chinese) and 89.0% (English); CycleGAN-EVC beats VAWGAN-EVC on MCD and listening tests.Paper · 2021-05-28ResT: An Efficient Transformer for Visual RecognitionResT is a multi-scale vision Transformer backbone. Its attention shrinks keys and values with a strided depth-wise convolution and mixes heads with a 1x1 convolution plus instance norm. Position comes from a depth-wise-conv gate in each patch embedding, which the paper says handles any input size. Author-reported: ResT-Small reaches 79.6% ImageNet-1k top-1.Paper · 2021-05-27Towards Understanding Knowledge DistillationTheory of distillation between linear classifiers, with soft labels and gradient flow. With n >= d the student recovers the teacher exactly. For n < d, a transfer-risk bound depends on the angle distribution, giving fast rates for margin and polynomial tasks. Author-reported experiments support data geometry, optimization bias and strong monotonicity.Paper · 2021-05-27YOLO5Face: Why Reinventing a Face DetectorTreats face detection as generic object detection: YOLOv5 gets a five-point landmark head trained with wing loss, a stem block, smaller SPP kernels, a stride-64 P6 output for large faces, and ShuffleNetV2 backbones for tiny models. Author-reported WiderFace validation (Easy/Medium/Hard): YOLOv5x6 96.67/95.08/86.55; YOLOv5n 93.61/91.54/80.53 at 1.7M parameters.Paper · 2021-05-24SuMa++: Efficient LiDAR-based Semantic SLAMSuMa++ adds semantic labels to surfel-based LiDAR SLAM (SuMa). RangeNet++ labels each laser point on a spherical projection, a flood-fill cleans the labels, and surfels whose class conflicts with a moving-object class lose stability. Semantic weights also enter the ICP residuals. On KITTI odometry training sequences, average translational error drops from 0.83% to 0.70% (author-reported).Paper · 2021-05-20Anabranch Network for Camouflaged Object SegmentationAnabranch Network (ANet) adds a camouflage classifier to a saliency FCN and multiplies its image-level probability into the pixel map, so the output is scaled by how likely camouflage is. Introduces CAMO, 1,250 annotated camouflage images. Author-reported on CAMO-COCO, ANet-SRM scores F-beta 0.826 and IoU 0.732, against 0.699 and 0.535 for the pre-trained SRM.Paper · 2021-05-20Data-Free Knowledge Distillation for Heterogeneous Federated LearningFedGen: the server trains a small conditional generator from the prediction layers of the user models, with no data and no proxy set. Its latent samples are broadcast and used as an inductive bias in local training, which speeds convergence under non-iid data. Author-reported: MNIST (Dirichlet alpha=0.05) 91.30% vs FedAvg 87.70%; EMNIST alpha=0.1 72.15% vs 66.21%; CelebA r=5/10 89.70% vs 87.48%.Paper · 2021-05-14DialogSum: A Real-Life Scenario Dialogue Summarization DatasetDialogSum is a 13,460-dialogue English dataset of spoken, daily-life conversations, each with a human-written observer-perspective summary and a topic. A BART-large baseline reaches ROUGE-1 47.28 on its test set, against 35.91 for a Transformer (author-reported, averaged over three references). Human checks find errors in coreference, discourse relations and speaker intents.Paper · 2021-05-13Dynamic View Synthesis from Dynamic Monocular VideoRenders a dynamic scene from new viewpoints and times, given one monocular video and moving-object masks. A static NeRF fits unmasked pixels; a time-conditioned NeRF predicts 3D scene flow and a blend, with flow, depth and smoothness regularisers. Author-reported on 7 sequences: PSNR 24.74, LPIPS 0.118 (Li et al.: 23.99, 0.205).Paper · 2021-05-13Grad-TTS: A Diffusion Probabilistic Model for Text-to-SpeechGrad-TTS maps text to mel-spectrograms with a score-based diffusion decoder that denoises Gaussian noise centred on text-aligned encoder outputs; Monotonic Alignment Search gives the alignment. More Euler steps give better audio at higher cost. On LJSpeech, 10 steps reach MOS 4.38 (Tacotron2 4.32) at RTF 0.033 vs 0.075 for Tacotron2 (author-reported).Paper · 2021-05-12BertGCN: Transductive Text Classification by Combining GCN and BERTBertGCN builds a word-document graph over the whole corpus, starts document nodes from BERT embeddings and trains BERT and a two-layer GCN jointly, using a memory bank of all document embeddings. It mixes the GCN and BERT predictions with a weight lambda. Author-reported test accuracy with BERT-base: 89.3 (20NG), 98.1 (R8), 96.6 (R52), 72.8 (Ohsumed), 86.0 (MR).Paper · 2021-05-12Segmenter: Transformer for Semantic SegmentationSegmenter is a convolution-free semantic segmentation model: a ViT encoder over image patches, then either a per-patch linear classifier or a 2-layer mask transformer whose learnable class embeddings produce one mask per class, upsampled bilinearly. Author-reported: 53.63% mIoU on ADE20K val (Seg-L-Mask/16, multi-scale), 59.0% on Pascal Context, 81.3% on Cityscapes.Paper · 2021-05-09Conformer: Local Features Coupling Global Representations for Visual RecognitionHybrid backbone with a ResNet-style CNN branch and a ViT-style transformer branch, coupled in every block by a Feature Coupling Unit that passes features both ways. Author-reported: Conformer-S reaches 83.4% top-1 on ImageNet-1k (DeiT-B 81.8%). Conformer-S/32 beats ResNet-101 by 3.7 box AP (Faster R-CNN) and 3.6 mask AP (Mask R-CNN) on COCO.Paper · 2021-05-06GraphFormers: GNN-nested Transformers for Representation Learning on Textual GraphNests a small GNN inside every layer of a pretrained language model, so each node's text is encoded with its neighbours' context. Adds span-masked warm-up training and cacheable unidirectional graph attention. Headline: higher P@1 than cascaded PLM+GNN baselines on Product, DBLP and Wiki (DBLP 0.7267 vs 0.6934; author-reported).Paper · 2021-05-05MOS: Towards Scaling Out-of-distribution Detection for Large Semantic SpaceSplits the 1,000 ImageNet classes into 8 semantic groups, adds an "others" class to each, and fine-tunes only the last layer with group softmax. The MOS score is the negated minimum "others" probability over groups. On four curated OOD sets (author-reported) it gives average FPR95 39.97 against 54.30 for KL matching, at 3.2 vs 20.6 min test time.Paper · 2021-05-05SeaDronesSee: A Maritime Benchmark for Detecting Humans in Open WaterDrone benchmark of swimmers, floaters, life jackets and boats in open water, with per-frame altitude and viewing-angle metadata. Defines detection and single- and multi-object tracking tasks, scored on a withheld-label server. Author-reported: best detector AP 30.4; best tracker MOTA 71.9 (Tracktor++).Paper · 2021-05-04Challenges and Outlook in Robotic Manipulation of Deformable ObjectsSurvey and position paper on robot manipulation of deformable objects. It reviews grippers, sensing, modelling, planning and control, and lists open challenges in each. It proposes no method and runs no experiments. Its only data is a poll of 31 researchers rating each area's importance and maturity, with sensing rated most promising.Paper · 2021-05-03Graph Learning: A SurveySurvey of graph learning, machine learning on graphs, sorted into four families: graph signal processing, matrix factorization, random walks and deep learning. Covers applications in text, images, science, knowledge graphs and combinatorial optimization, then open issues. It runs no experiments and reports no benchmark numbers, so it maps the field rather than ranking methods.Paper · 2021-05-03Hybrid IntelligencePosition paper (catchword, no experiments) defining Hybrid Intelligence: humans and AI with complementary strengths jointly perform tasks and learn from each other, aiming at results neither reaches alone. Describes two loops (AI in the loop of human intelligence, humans in the loop of AI) and three research directions: trust, governance and digital work. No method, measurement or result.Paper · 2021-05-01FedProto: Federated Prototype Learning across Heterogeneous ClientsFedProto is federated learning where clients exchange per-class mean embeddings (prototypes) instead of weights. The server averages them per class; each client adds a loss pulling its embeddings toward the global prototypes. Works with different label sets and architectures. 20 clients: 84.49% on CIFAR10 (n=3) vs 81.72% for FedAvg, sending ~41k instead of 235M parameters per round.Paper · 2021-04-30Editable Free-viewpoint Video Using a Layered Neural RepresentationST-NeRF: each performer and the background is a space-time neural radiance field with its own deformation MLP, and the layers are composited by object-aware volume rendering. Trained from 16 cameras (up to 180 degrees), it allows moving, duplicating, retiming and fading layers. Author-reported PSNR 33.22 on their captured data vs 28.26 for the best baseline (NeRF-T).Paper · 2021-04-30Explaining a Series of Models by Propagating Shapley ValuesDeepSHAP explains a chain of models (deep, tree, linear) by passing Shapley-style attributions backward through it, rescaling at each link, and averaging over a baseline distribution. Author-reported: about an order of magnitude faster than model-agnostic methods, with comparable ablation scores, and it can explain a distributed stack without sharing its models.Paper · 2021-04-29Emerging Properties in Self-Supervised Vision TransformersDINO trains a ViT with no labels by making a student match the softmax outputs of a momentum (EMA) teacher across augmented views, with teacher centering and sharpening to prevent collapse. Author-reported: 78.3% k-NN top-1 on ImageNet with ViT-S/8 and 80.1% linear top-1 with ViT-B/8.Paper · 2021-04-29Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine TranslationProfessional translators mark every error span in WMT20 English-German and Chinese-English outputs, with a category and severity, and the marks become a segment score (Major 5, Minor 1). Author-reported: MQM reorders the WMT crowd ranking, puts all human translations above all MT, and most metrics correlate better with MQM than with WMT scores.Paper · 2021-04-27BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and AlignmentUpgrades BasicVSR for video super-resolution with second-order grid propagation (two previous frames, two backward/forward rounds) and flow-guided deformable alignment, where optical flow sets the base offsets and a learned residue refines them. Author-reported: 32.39 dB PSNR on REDS4 (BI, x4) with 7.3M parameters and 77 ms, against 31.42 dB for BasicVSR.Paper · 2021-04-27Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemsPhysics-aware training (PAT) trains a chain of real physical systems with backpropagation. The forward pass runs on the hardware, and gradients come from a differentiable digital model. Shown on optical, mechanical and analog-circuit networks. Author-reported: 93% vowel accuracy with PAT versus about 40% for in silico training.Paper · 2021-04-27Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality MetricsFRANK benchmarks factuality metrics for abstractive summarization: 2250 CNN/DM and XSum summaries from nine systems, labelled sentence by sentence by crowd workers under a seven-category error typology. Common metrics correlate weakly with human judgement. Author-reported: best Pearson 0.27 (BERTScore P), best Spearman 0.30 (FactCC).Paper · 2021-04-24A Survey of Modern Deep Learning based Object Detection ModelsSurvey of deep-learning object detectors: datasets and metrics, backbones from AlexNet to EfficientNet, two-stage and single-stage detectors, and lightweight networks. It compiles author-reported COCO/VOC AP and FPS for 17 detectors and top-1, latency and FLOPs for 9 lightweight models. The numbers are copied from source papers, not reproduced.Paper · 2021-04-22All Tokens Matter: Token Labeling for Training Better Vision TransformersToken labeling adds a dense loss to vision transformers: each patch token is trained against its own label from a pretrained NFNet-F6 annotator, and MixToken mixes patch embeddings CutMix-style. Author-reported ImageNet top-1 is 84.4% for LV-ViT-S (26M) and 86.4% for LV-ViT-L at 512 px (about 150M).Paper · 2021-04-22ImageNet-21K Pretraining for the MassesCleans ImageNet-21K (drops classes under 500 images, fixes a 50-image validation split, resizes to 224 px) and pretrains with semantic softmax: one softmax per WordNet depth, each image trained at every depth its ancestor labels reach, with depth losses reweighted. Author-reported: beats ImageNet-1K pretraining on all 30 model-dataset cells tested.Paper · 2021-04-22LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and MappingLVI-SAM fuses a monocular camera, a lidar and an IMU in one iSAM2 factor graph. A visual-inertial and a lidar-inertial sub-system seed and check each other, and DBoW2 loop candidates are verified by ICP. On self-gathered Jackal and Handheld datasets it reports the lowest RMSE against GPS; most lidar-based baselines fail on Handheld.Paper · 2021-04-22Motion Representations for Articulated AnimationAnimates articulated objects such as human bodies from a driving video, without labels. Each unsupervised region's centre, shape and orientation come from the PCA of its heatmap, not regressed parameters. A global affine term absorbs camera motion; a separate network splits source shape from driving pose. Author-reported: lower errors than FOMM, and 96.6% user preference over FOMM on TED-talks.Paper · 2021-04-21BADet: Boundary-Aware 3D Object Detection from Point CloudsAdds a graph refinement stage to two-stage LiDAR 3D detection. Each region proposal is a node in a radius graph whose state fuses voxel, BEV-pixel and raw-point features (RFA). Max-aggregated message passing with learned offsets lets each proposal recover object boundaries. Author-reported KITTI test Car moderate: 81.61 AP3D, 91.32 APBEV; nuScenes val: 47.65 mAP, 58.84 NDS.Paper · 2021-04-20Gradient Matching for Domain GeneralizationFish approximates an inter-domain gradient matching objective (IDGM) with a first-order update. Each meta-step takes one plain gradient step on a clone of the model for every training domain, then moves the weights toward the clone. Author-reported: best on 4 of 6 Wilds datasets, 67.1% average on DomainBed (Coral 67.2%, ERM 67.0%).Paper · 2021-04-20UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View ReconstructionUNISURF models a surface as a neural occupancy field and renders it two ways: surface rendering at the first ray crossing, and volume rendering over a sample window around that crossing whose width shrinks during training. This needs no object masks. On 15 DTU scans it reports mean Chamfer 1.02 without masks, against 0.90 for IDR with masks and 1.49 for NeRF-derived meshes.Paper · 2021-04-20VideoGPT: Video Generation using VQ-VAE and TransformersVideoGPT compresses a video into discrete latent codes with a 3D-convolutional VQ-VAE that uses axial attention, then fits a GPT-style transformer prior over those codes and decodes sampled codes back to pixels. Author-reported: FVD 103.3 on BAIR Robot Pushing and Inception Score 24.69 on UCF-101.Paper · 2021-04-18Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled CorpusDocuments C4, the cleaned April 2019 Common Crawl corpus behind T5, in three views: sources (domains, dates, IP countries), contents (machine-translated patents, benchmark test examples, sentiment gaps for Jewish and Arab mentions) and removed text (a word blocklist hits African American English and sexual-orientation topics). Advises against blocklists. Author-reported; no model.Paper · 2021-04-18Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order SensitivityFew-shot in-context learning depends on the order of the demonstrations. The method samples unlabelled probe inputs from the LM for each candidate order and keeps the four orders whose predicted label distribution has the highest entropy (GlobalE). Author-reported: about 13% average relative gain over unselected prompts on 11 text classification tasks.Paper · 2021-04-17TransVG: End-to-End Visual Grounding with TransformersTransVG grounds a text query to a box with transformer encoders only: a DETR-initialised visual branch and a BERT branch feed one fusion stack with a learnable [REG] token, whose output regresses the four box coordinates directly, with no proposals or anchors. Author-reported top-1 accuracy at IoU 0.5 with ResNet-101: 70.73 on ReferItGame test, 79.10 on Flickr30K test, 78.35 on RefCOCO testB.Paper · 2021-04-16Surface Form Competition: Why the Highest Probability Answer Isn't Always RightZero-shot multiple choice with GPT-2 and GPT-3 by scoring each option as log P(option | question) minus log P(option | a short task-specific domain premise), called Domain Conditional PMI. It removes the penalty on answers that have many valid wordings. Author-reported: best or tied on 62.5% of 16 splits for GPT-3 175B, against 18.75% for length-normalised log-likelihood.Paper · 2021-04-15A survey of active learning algorithms for supervised remote sensing image classificationReviews and tests active learning heuristics for remote sensing image classification: committee (entropy query-by-bagging, adaptive maximum disagreement), large-margin (margin sampling, MCLU, diversity variants) and posterior-probability (KL-max, Breaking Ties) methods on three datasets. With SVMs, large-margin heuristics with a diversity term suit best; results are plots only.Paper · 2021-04-15Contrastive Learning with Stronger AugmentationsCLSA adds a distribution-matching loss to MoCo-style contrastive pretraining. A strongly augmented view must match the similarity profile its weak twin gives over a key queue, not the twin itself. ResNet-50 ImageNet linear top-1 at 200 epochs: 69.4% vs MoCo v2 67.5%. Multi-crop CLSA* reaches 76.2% at 800 epochs vs 76.5% supervised. Author-reported.Paper · 2021-04-15HIVE-COTE 2.0: a new meta ensemble for time series classificationMeta ensemble for time series classification with four modules (STC, DrCIF, Arsenal, TDE), each on its own representation, weighted by estimated train accuracy. Author-reported: significantly more accurate than the prior state of the art on 112 UCR and 26 UEA problems, at about 340 sequential hours for one UCR resample.Paper · 2021-04-14Learning How to Ask: Querying LMs with Mixtures of Soft PromptsLearns cloze prompts for querying masked language models by gradient descent. Prompt tokens become continuous vectors, each layer gets an additive perturbation, and a mixture of such prompts is trained with EM. On T-REx with BERT-large, the mixture reaches P@1 51.6 versus 39.4 for LPAQA (author-reported, reimplemented baseline).Paper · 2021-04-14TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding LearningTrains a sentence encoder without labels: each sentence loses 60% of its words, the encoder turns the damaged text into one CLS vector, and a weight-tied Transformer decoder rebuilds the original from that vector alone. Author-reported: best unsupervised average on four domain tasks (55.2 vs 52.4 for CT) and up to 6.4 points over CT on SciDocs; also a strong pre-training step.Paper · 2021-04-13BARF: Bundle-Adjusting Neural Radiance FieldsBARF fits a NeRF and its camera poses jointly from imperfect or unknown poses. It starts with only the low-frequency positional-encoding bands and unmasks higher bands over training, a coarse-to-fine schedule. Author-reported: on 8 Blender scenes with perturbed poses, mean rotation error falls from 6.17 to 0.19 degrees and PSNR rises from 22.12 to 27.50 dB, near NeRF with true poses (29.40 dB).Paper · 2021-04-13Document-Level Event Argument Extraction by Conditional GenerationFrames event argument extraction as conditional generation: BART writes an event template with its slots filled from the whole document, copying only input tokens. Author-reported: WikiEvents argument classification Head F1 64.57 vs 56.16 for BERT-QA; RAMS Head F1 57.32 vs 49.7 for Two-step.Paper · 2021-04-13Lite-HRNet: A Lightweight High-Resolution NetworkLite-HRNet is a small high-resolution network for human pose estimation and semantic segmentation. It starts from ShuffleNet blocks inside HRNet, then replaces the costly 1x1 convolutions with conditional channel weights computed across channels and resolutions. Lite-HRNet-30 reaches 67.2 COCO val AP at 256x192 with 1.8M parameters and 0.31 GFLOPs (author-reported).Paper · 2021-04-13QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question AnsweringQA-GNN answers multiple-choice questions by joining a pretrained LM with a ConceptNet subgraph. The LM scores each KG node's relevance to the question, a QA-context node links to the topic entities, and an attention GNN reasons over both. Author-reported: 73.41% on the CommonsenseQA in-house test (RoBERTa-large alone 68.69%).Paper · 2021-04-12Machine learning and deep learningConceptual review, not a new method. Defines AI, machine learning and deep learning, contrasts explicit programming, shallow ML and deep learning as ways to build analytical models, and lists practical challenges: data bias and drift, accuracy versus cost, explainability and transfer learning. No experiments and no code.Paper · 2021-04-11Graph Representation Learning in BiomedicineReview of graph representation learning for biomedical networks. It groups methods into shallow node embeddings, graph neural networks and generative graph models, then maps them to molecules, genomics, drugs and patient records. It runs no experiments and reports no benchmark numbers. No official code.Paper · 2021-04-083D Shape Generation and Completion through Point-Voxel DiffusionPoint-Voxel Diffusion (PVD) applies DDPM-style denoising diffusion to 3D point clouds with a point-voxel CNN. One network does unconditional generation and shape completion, where the partial shape is held fixed and only missing points are denoised. Author-reported 1-NN on ShapeNet: best on all six generation cells (CD and EMD, Airplane, Chair, Car); best EMD but not CD for completion.Paper · 2021-04-05An Empirical Study of Training Self-Supervised Vision TransformersTrains Vision Transformers with contrastive Siamese self-supervision (MoCo v3) and finds training unstable, hiding accuracy loss. Freezing the random patch projection stabilises training. ViT-B/16 reaches 76.7% linear-probing top-1 on ImageNet and ViT-BN-L/7 reaches 81.0%, author-reported.Paper · 2021-04-01Group-Free 3D Object Detection via TransformersGroup-Free 3D detects objects in point clouds without grouping points to candidates: each candidate attends to all points through stacked transformer decoder layers, refines its box-based position encoding per stage, and merges predictions from all stages with NMS. Author-reported: 67.3 mAP@0.25 and 48.9 mAP@0.5 on ScanNet V2 (6 layers, 256 candidates), and 63.0 and 45.2 on SUN RGB-D.Paper · 2021-04-01LoFTR: Detector-Free Local Feature Matching with TransformersLoFTR matches two images without a keypoint detector. A CNN-FPN gives coarse (1/8) and fine (1/2) features; linear-attention Transformer layers make them scene-aware; dual-softmax with mutual-nearest selection gives coarse matches, refined to sub-pixel in 5x5 windows. Author-reported: 40.8 pose AUC@10 on ScanNet and 69.19 on MegaDepth (dual-softmax).Paper · 2021-04-01NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular VideoNeuralRecon predicts a sparse TSDF volume for each fragment of 9 key frames in one network pass. It unprojects image features into 3D, decodes them coarse-to-fine with sparse 3D convolutions, and fuses them with earlier fragments via a ConvGRU. Author-reported: F-score 0.562 on ScanNet (Atlas split) at 30 ms per key frame on an RTX 2080 Ti.Paper · 2021-04-01PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and RelightingPhySG recovers an SDF geometry (MLP), a spatially varying diffuse albedo (MLP), one monochrome specular spherical-Gaussian (SG) lobe and a 128-SG environment map from posed RGB images of a glossy object under static light. Shading is a closed-form SG integral, so material and light can be edited and relit afterwards. Author-reported: synthetic novel-view PSNR 35.93 and normal error 2.528 degrees.Paper · 2021-04-01Putting NeRF on a Diet: Semantically Consistent Few-Shot View SynthesisDietNeRF fits a neural radiance field to a few posed photos, adding a semantic consistency loss. Every 10 iterations it renders a random pose and matches the frozen CLIP ViT embedding of that render to a training view's embedding. Author-reported: on 8-view Realistic Synthetic scenes, 23.15 PSNR and 74.9 FID versus 14.93 and 228.1 for NeRF; a short pixel-loss fine-tune gives 23.59 PSNR.Paper · 2021-03-31ArtFlow: Unbiased Image Style Transfer via Reversible Neural FlowsUniversal style transfer that avoids content leak. A Glow-style reversible flow (two blocks of eight flows) maps content and style images to latent features, AdaIN or WCT transfers the style there, and the reverse pass returns an image. Author-reported: SSIM 0.45 against 0.29 for AdaIN with comparable stylization, and no visible content leak after 20 rounds for its AdaIN and WCT variants.Paper · 2021-03-31DER: Dynamically Expandable Representation for Class Incremental LearningDER does class-incremental image classification by freezing the feature extractors of earlier steps and adding a new one per step, concatenating all features before one linear classifier. It adds an auxiliary new-vs-old loss, channel-mask pruning (described, not released), and a balanced classifier retrain. Reports 72.05% average accuracy on CIFAR-100 with 50 steps (prior best 64.32%).Paper · 2021-03-31StyleCLIP: Text-Driven Manipulation of StyleGAN ImageryEdits StyleGAN2 images from CLIP text prompts in three ways: per-image latent optimisation (98 s inference on a GTX 1080Ti), a per-prompt latent mapper (75 ms inference, 10-12 h training), and a global direction in style space S chosen by per-channel CLIP relevance. Results are visual comparisons only; timings are author-reported.Paper · 2021-03-30Distribution Alignment: A Unified Framework for Long-tail Visual RecognitionDistribution Alignment retrains only the classifier of a two-stage long-tail model: a learned per-class scale and offset, blended with the original scores by an input-dependent confidence, trained with inverse-frequency class weights raised to a power rho. Author-reported: 52.9% per-class top-1 on ImageNet-LT (ResNet-50, cosine head); 47.8 mIoU on ADE20K.Paper · 2021-03-30Rethinking Spatial Dimensions of Vision TransformersPooling-based Vision Transformer (PiT) adds ResNet-style spatial reduction to ViT. Two depth-wise convolution pooling layers shrink the token grid (27, 14, 7 for the tiny to small models) while channels grow with the head count. Author-reported: PiT-S reaches 80.9% ImageNet top-1 with the DeiT recipe vs 79.8% for ViT-S, with fewer FLOPs.Paper · 2021-03-29CvT: Introducing Convolutions to Vision TransformersCvT adds convolutions to ViT in two places: an overlapping convolutional token embedding that builds a three-stage hierarchy, and depthwise-convolutional query, key and value projections whose keys and values are subsampled with stride 2. It drops position embeddings. Author-reported: 81.6% ImageNet-1k top-1 with 20M parameters (CvT-13), and 87.7% with CvT-W24 pre-trained on ImageNet-22k.Paper · 2021-03-29Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder TransformersExplains any attention-based Transformer, including bi-modal co-attention and encoder-decoder (DETR-style) models. Attention maps are weighted by the target gradient, positive parts kept, heads averaged, and relevancy propagated layer by layer. Weakly supervised DETR segmentation: AP 13.1 vs 7.2 for the best baseline (author-reported).Paper · 2021-03-29RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective WhiteningInstance selective whitening (ISW): a training-only loss that penalizes only the feature-covariance entries that shift under color and blur augmentation, found by k-means on their variance, in three instance-norm layers of DeepLabV3+. Trained on GTAV with ResNet-50, mIoU on Cityscapes, BDD-100K and Mapillary is 36.58, 35.20 and 40.33 (baseline 28.95, 25.14 and 28.18). Author-reported.Paper · 2021-03-26Character Controllers Using Motion VAEsA Motion VAE (MVAE) is a conditional VAE that predicts the next human pose from the current one, with a 32-dimensional latent. Sampling the latent gives plausible random motion, and RL policies that output the latent steer it to targets, joystick speed and heading, paths and maze exploration. Author-reported: foot skate 0.07 to 0.30 cm/frame vs 0.10 in mocap.Paper · 2021-03-25Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered ScenesContact-GraspNet takes a depth point cloud of a cluttered scene and predicts, for each point, whether it can be a parallel-jaw contact. Each contact comes with a grasp rotation and width, so the 6-DoF grasp is reduced to 4 DoF. Trained on simulated ACRONYM grasps. Author-reported: 90.20% grasp success on 51 unseen objects, against 80.39% for the best baseline.Paper · 2021-03-25KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPsKiloNeRF replaces NeRF's single large MLP with a uniform 3D grid of up to 16x16x16 tiny MLPs (4 hidden layers, 32 units), each covering one cell. It is trained by distilling a pretrained NeRF, then fine-tuned on the images, and rendered with empty space skipping and early ray termination. Authors report quality on par with NeRF at 1258-2167x faster rendering.Paper · 2021-03-25Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsSwin Transformer is a hierarchical vision Transformer. Self-attention runs inside local 7x7 windows, and the window grid shifts by half a window in every other block so neighbouring windows exchange information. Patch merging halves resolution across four stages. Author-reported: 81.3% ImageNet-1K top-1 for Swin-T, 58.7 box AP on COCO test-dev and 53.5 mIoU on ADE20K val.Paper · 2021-03-24Learning Salient Boundary Feature for Anchor-free Temporal Action LocalizationAFSD, an anchor-free temporal action localizer. A 1D pyramid predicts start and end distances at each location. A refinement step max-pools the most salient start and end features of each proposal (boundary pooling), and consistency losses regularise that pooling. Author-reported: 52.0 average mAP on THUMOS14 and 34.4 on ActivityNet 1.3.Paper · 2021-03-23Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayersBCNet adds an occlusion branch to the mask head of a detect-then-segment model. A first graph (non-local) layer predicts the occluding object's masks and contours, and a second graph layer, guided by it, predicts the partly occluded object. Author-reported: 39.8 mask AP on COCO test-dev with Faster R-CNN and ResNet-101.Paper · 2021-03-23Generalizing Face Forgery Detection with High-frequency FeaturesTwo-stream face forgery detector: an Xception RGB stream plus SRM high-pass noise features taken at several scales, linked by residual-guided spatial attention and dual cross-modality attention. Trained on FF++ (HQ), it beats Xception and Face X-ray on DFD, DFDC, CelebDF and DF1.0 (AUC), and reaches 0.794 on CelebDF against 0.644 for FFD (author-reported).Paper · 2021-03-23Scaling Local Self-Attention for Parameter Efficient Visual BackbonesHaloNet builds image backbones from local 2D self-attention. Blocked local attention with haloing (unmasked, non-centred windows) and strided attention downsampling make it fast enough to train. Author-reported: H7 reaches 84.9% top-1 on ImageNet when trained from scratch, and the family matches EfficientNet at equal parameter counts.Paper · 2021-03-22An Experimental Review on Deep Learning Architectures for Time Series ForecastingBenchmarks seven deep architectures (MLP, Elman RNN, LSTM, echo state network, GRU, 1D CNN, TCN) for univariate forecasting on 12 public datasets (over 50,000 series), grid-searching over 38,000 models scored by WAPE. Author-reported: LSTM gives the most accurate best-case forecasts; CNN and LSTM rank best on mean WAPE and its spread; CNN gives the best speed/accuracy trade-off.Paper · 2021-03-22BERT: A Review of Applications in Natural Language Processing and UnderstandingSurvey of BERT and its uses in NLP: pretraining and retraining strategies, text classification, extractive annotation, BERTScore, TextFooler-style attacks, interlingual and domain models (SciBERT, BioBERT). Headline: fine-tuned SciBERT beats fine-tuned BERT-Base on all 12 rows of its Table 1. Numbers are author-reported by the cited papers.Paper · 2021-03-22Quality at a Glance: An Audit of Web-Crawled Multilingual DatasetsManual audit of 205 language-specific subsets of CCAligned, ParaCrawl, WikiMatrix, OSCAR and mC4, about 100 sampled sentences each, with an error taxonomy. Author-reported: 15 subsets have no correct sentence, 87 are under 50% correct, and 83 carry nonstandard or wrong language codes. Recommends sentence-level checks before use.Paper · 2021-03-22SSD: A Unified Framework for Self-Supervised Outlier DetectionSSD trains a ResNet-50 on unlabeled in-distribution images with a SimCLR-style contrastive loss, then scores inputs by Mahalanobis distance of their normalised features to the training features. Author-reported average AUROC over four CIFAR-10/100 OOD pairs: 88.7 without labels, 92.5 with five OOD samples per class.Paper · 2021-03-20Interpretable Machine Learning: Fundamental Principles and 10 Grand ChallengesSurvey of inherently interpretable machine learning. It states five principles, such as that interpretable models need not cost accuracy, then lists 10 open technical challenges: optimal sparse trees, scoring systems, GAMs, case-based reasoning, neural disentanglement, dimension reduction, physics constraints, Rashomon sets and interpretable RL. No experiments of its own.Paper · 2021-03-18GPT Understands, TooP-Tuning adds trainable continuous prompt vectors, passed through a small LSTM or MLP, next to the words of a hand-written prompt. The language model is frozen for LAMA knowledge probing and tuned for SuperGLUE. Author-reported: LAMA-34k P@1 with BERT-base rises from 43.3 (AutoPrompt) to 48.3, and the spread across manual prompts shrinks.Paper · 2021-03-18ICDAR2019 Competition on Scanned Receipt OCR and Information ExtractionCompetition report for ICDAR 2019 on scanned receipts: 1000 receipt images (600 trainval, 400 test) and three tasks: text localisation, word OCR and key-field extraction, ranked by F1. Author-reported top scores: Hmean 98.59 (localisation), 96.43 (OCR), 90.49 (key-field extraction). No method is proposed.Paper · 2021-03-17A Comprehensive Survey of Scene Graphs: Generation and ApplicationSurvey of scene graph generation, which detects objects in an image and predicts the relations between them as subject-predicate-object triplets. It groups methods by model family (CRF, TransE, CNN, RNN/LSTM, GNN) and by prior knowledge, reviews applications and datasets, and tabulates author-reported VG150 results (best SGCls R@100 47.1, BGT-Net). It has no method or experiments of its own.Paper · 2021-03-17OGB-LSC: A Large-Scale Challenge for Machine Learning on GraphsBenchmark paper for the KDD Cup 2021 OGB-LSC challenge: three large graph datasets (MAG240M node classification, WikiKG90M link prediction, PCQM4M graph regression) with baselines and a summary of winning techniques. Expressive models beat simple scalable ones. Author-reported best test scores: 69.42% accuracy (R-GAT), 0.8637 MRR (ComplEx-Concat), 0.1494 eV MAE (GIN-virtual).Paper · 2021-03-17PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive LearningPredRNN is a stacked recurrent net whose memory flows zigzag up through the layers and forward in time, with two memory cells kept apart by a cosine decoupling loss. Reverse scheduled sampling trains the encoder on long-term dynamics. PredRNN-V2 reports the best MSE on fixed-set Moving MNIST (48.4) and top scores on KTH, radar, Traffic4Cast and BAIR (author-reported).Paper · 2021-03-17Pros and Cons of GAN Evaluation Measures: New DevelopmentsSurvey of GAN evaluation measures that appeared after the author's 2019 review: FID variants, FVD, perceptual path length, precision-recall extensions, classifier-based, data-copying and human-judged scores, and links to fairness and deepfake detection. It runs no new experiments; the numbers are results cited from other papers.Paper · 2021-03-16Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery DetectionFace forgery detector: an Xception RGB branch fused with a learned branch over YCbCr 8x8 DCT coefficients, trained with softmax loss plus a single-center loss. That loss pulls natural faces to one updatable center and pushes manipulated faces at least a margin farther away. Author-reported video AUC 0.924 and pAUC(0.1) 0.810 on FF++ c40, against 0.861 and 0.652 for their baseline.Paper · 2021-03-13A review of machine learning in processing remote sensing data for mineral explorationReview of machine learning applied to satellite, airborne and ground-based remote sensing data for mapping lithology, alteration, structures and minerals in ore exploration. Covers dimensionality reduction, classification, clustering, regression and deep learning. The paper runs no experiments of its own; the few numbers it gives come from cited studies.Paper · 2021-03-13ReDet: A Rotation-equivariant Detector for Aerial Object DetectionReDet makes an oriented-object detector rotation-equivariant. Its backbone is a ResNet-50 with FPN, equivariant to 8 discrete rotations, and RiRoI Align pools rotated RoIs after aligning them in space and orientation. Author-reported: 80.10 mAP on DOTA-v1.0 (multi-scale), 76.80 on DOTA-v1.5 and 90.46 on HRSC2016 (VOC2007), with a 12 Mb backbone against 103 Mb.Paper · 2021-03-12UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color SpaceUIEC^2-Net enhances underwater images with three CNN blocks: an RGB block that removes noise and colour cast, an HSV block that predicts piecewise-linear curves for brightness and saturation, and an attention block that blends the two outputs per pixel. Trained end-to-end with L1, SSIM, HSV and perceptual losses. Author-reported PSNR on real-world images: 24.57 dB, best among compared methods.Paper · 2021-03-10FSCE: Few-Shot Object Detection via Contrastive Proposal EncodingFSCE fine-tunes a Faster R-CNN with a frozen ResNet-101 and adds a contrastive head. Its supervised contrastive loss pulls same-class RoI proposals together and pushes other classes apart, using only proposals with high IoU. Author-reported: best or tied nAP50 on all three PASCAL VOC novel splits for 1 to 10 shots, up to +8.8 on split 3 at 10 shots, and +2.7 nAP on COCO at 30 shots.Paper · 2021-03-09PointDSC: Robust Point Cloud Registration using Deep Spatial ConsistencyPointDSC removes outlier correspondences for 3D point cloud registration. A nonlocal attention block weights feature similarity by the length consistency of point pairs, and seeded neural spectral matching fits a rigid transform to each kNN subset. On 3DMatch with FCGF it reports 93.28% registration recall at 0.09 s per pair (author-reported).Paper · 2021-03-09Pretrained Transformers as Universal Computation EnginesFreezes the self-attention and feedforward blocks of a GPT-2 transformer pretrained on natural language and trains only linear input and output layers, positional embeddings and layer norms on bit, image, ListOps and protein tasks. Author-reported: 100% on both bit tasks, 98.0% MNIST, 72.1% CIFAR-10 (36 layers), comparable to fully trained transformers.Paper · 2021-03-08Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive ModelsSurvey, not a new method. Compares energy-based, variational, adversarial, autoregressive and normalizing-flow generative models on training and sampling cost, exact density, and CIFAR-10 FID and NLL, and lists the core equations and trade-offs of each family. Its numbers are compiled from cited works, not re-measured.Paper · 2021-03-08Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to DoFinds a moral direction in BERT-large sentence embeddings: the first principal component of verb embeddings in question templates. Its scores track human yes/no judgements (r = 0.78 and 0.79, author-reported). As a filter after top-k and top-p sampling, it lowers toxicity in GPT-2 generations on RealToxicityPrompts.Paper · 2021-03-08Personalized Federated Learning using HypernetworksPersonalized federated learning where a server-side hypernetwork maps a trainable embedding per client to that client's network weights. Clients run K local SGD steps and return the weight change, which is backpropagated into the hypernetwork, so the hypernetwork is never transmitted. Author-reported: 90.83% on CIFAR10 with 10 clients; below Per-FedAvg on Omniglot.Paper · 2021-03-07Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and ApplicationsSystematic review (PRISMA with LDA screening) of model-agnostic counterfactual explanations for explainable AI. Proposes a seven-category taxonomy of the surveyed algorithms and argues they are not grounded in causal theory, so they may not give causability. Only RECOURSE is marked causal in its summary table. The paper runs no experiments of its own.Paper · 2021-03-07Student-Teacher Feature Pyramid Matching for Anomaly DetectionDistils an ImageNet-pretrained ResNet-18 teacher into a same-architecture student by matching L2-normalised features at three pyramid levels (conv2_x to conv4_x) on anomaly-free images. The anomaly map is the element-wise product of upsampled per-level distance maps. Author-reported on MVTec AD: 0.955 image AUC-ROC, 0.970 pixel AUC-ROC, 0.921 PRO.Paper · 2021-03-06Simultaneously Localize, Segment and Rank the Camouflaged ObjectsLocalizes, segments and ranks camouflaged objects in one ResNet-50 network. A fixation branch finds the region that makes an object visible, reverse attention feeds it to a mask branch, and a Mask R-CNN-style head ranks camouflage level from eye-tracking delays. Author-reported: COD10K S-measure 0.793 vs SINet 0.776 with COD10K+CAMO training. Released code has no ranking branch.Paper · 2021-03-06WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face RecognitionBuilds WebFace260M (4M names, 260M web faces) and its cleaned subset WebFace42M (2M identities, 42M faces) with CAST, a self-training cleaning loop. Adds the FRUITS time-limited face test and a test set. Author-reported: ArcFace ResNet-100 on WebFace42M reaches 97.70% TAR@FAR=1e-4 on IJB-C, and ranks 3rd of 430 on NIST-FRVT.Paper · 2021-03-05Unbalanced minibatch Optimal Transport; applications to Domain AdaptationReplaces the balanced optimal transport loss computed on each minibatch with unbalanced OT (KL-penalised marginals, entropic regularisation) and proves unbiasedness, concentration and gradient properties. Its JUMBOT domain-adaptation method reports 70.0% average on Office-Home and 98.0% on digits (author-reported).Paper · 2021-03-04Coordinate Attention for Efficient Mobile Network DesignCoordinate attention replaces the global pooling of channel attention with two 1D poolings along height and width, encodes both with one shared 1x1 conv, and gates the input per row and per column. Plugged into MobileNetV2 it reaches 74.3% ImageNet top-1 vs 73.5% for SE and 72.3% for the baseline (author-reported).Paper · 2021-03-04DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle NetworksRenders a NeRF-style scene with 2 to 16 shading samples per ray. A depth oracle MLP is evaluated once per ray and classifies log-spaced depth bins; its targets are filtered in image and depth space, and samples are placed by inverting its CDF. Author-reported: 31.14 dB mean PSNR at 4 samples per ray, against 30.52 dB for NeRF at 256 samples.Paper · 2021-03-03Random Feature AttentionRFA replaces softmax attention with a random-Fourier-feature approximation, so attention costs time and memory linear in sequence length. An optional learned gate adds recency bias. Author-reported: close to Transformer baselines on WikiText-103, WMT14, IWSLT14 and Long Range Arena, and about 1.8-1.9x faster decoding on MT.Paper · 2021-03-03Towards Open World Object DetectionDefines open world object detection and proposes ORE, a Faster R-CNN variant that clusters RoI features with a contrastive loss, pseudo-labels high-objectness background boxes as unknown, and flags unknowns with an energy score fitted by Weibull distributions. Author-reported: lower Wilderness Impact than Faster R-CNN, and Task 2 mAP over both known sets of 38.98 vs 14.91.Paper · 2021-03-02A Survey of Deep Learning Techniques for Weed Detection from ImagesSurvey of 70 deep-learning papers (2016 to August 2020) on finding weeds in crop images. It sorts them by data acquisition, dataset preparation, detection approach, learning method, architecture and metrics, and lists 19 public datasets. The reported scores come from different crops, datasets and metrics, so they are not directly comparable.Paper · 2021-03-02Generalizing to Unseen Domains: A Survey on Domain GeneralizationSurvey of domain generalization: train on source domains, predict on an unseen target domain. Sorts methods into data manipulation, representation learning and learning strategies, reviews the theory, and benchmarks seven methods in DeepDG (ResNet-18). Best author-reported averages: 82.0 on PACS (RSC, CORAL), 63.7 on Office-Home (RSC).Paper · 2021-03-01Accounting for Variance in Machine Learning BenchmarksBenchmark scores vary with data split, seed and hyperparameter search, and the data split matters most. Proposes randomising all sources except hyperparameter search (biased, about 51x cheaper than the ideal estimator) and deciding with P(A>B) >= 0.75 plus a bootstrap interval. Results are simulations and five case studies, author-reported.Paper · 2021-03-01Categorical Depth Distribution Network for Monocular 3D Object DetectionMonocular 3D detector: predicts a per-pixel categorical distribution over depth bins, weights ResNet-101 features by it, lifts them into a voxel grid, collapses to BEV and runs a PointPillars-style head, trained end-to-end. Author-reported: first among published monocular methods on KITTI test (Car moderate AP R40 13.41%).Paper · 2021-03-01Heterogeneity for the Win: One-Shot Federated ClusteringOne-shot federated k-means (k-FED): each device runs a local Lloyd-style clustering once and sends only its few local centers; the server groups the centers with a farthest-point seeding and one assignment round. With at most sqrt(k) clusters per device, cluster pairs that never share a device need only about k^(1/4) separation. Author-reported: near-centralized accuracy on synthetic mixtures.Paper · 2021-03-01Towards Personalized Federated LearningSurvey of personalized federated learning (PFL) for non-IID client data. It sorts PFL methods into two strategies: adapting one global model per client (data-based and model-based) and learning a separate model per client (architecture-based and similarity-based). It reviews non-IID benchmarks and metrics and lists open directions. No new experiments; the few numbers are quoted from cited work.Paper · 2021-02-28A Survey on Deep Semi-supervised LearningSurvey of deep semi-supervised learning for image classification. Sorts methods into generative (semi-supervised GANs and VAEs), consistency regularization, graph-based, pseudo-labeling and hybrid (MixMatch, FixMatch and similar), writes out their losses in shared notation, tabulates their perturbations and augmentations, and lists open problems. No experiments of its own.Paper · 2021-02-27Anomaly Detection on Attributed Networks via Contrastive Self-Supervised LearningCoLA: unsupervised node anomaly detection on attributed networks. A one-layer GCN with a bilinear discriminator learns to match each node with its sampled local subgraph, not with a random other node's subgraph. A node's score is the mean over 256 rounds of negative minus positive score. Author-reported ROC-AUC 0.7513 to 0.9512 on seven benchmarks with injected anomalies, best in every column.Paper · 2021-02-27Graph Self-Supervised Learning: A SurveySurvey of self-supervised learning on graphs. One encoder-decoder formulation sorts methods into generation, auxiliary property, contrast and hybrid pretext tasks, with three training schemes. Compiles reported node and graph classification accuracies; the best Cora and Citeseer rows are hybrid methods, but the numbers come from different sources.Paper · 2021-02-26Gradient Descent on Neural Networks Typically Occurs at the Edge of StabilityFull-batch gradient descent on neural networks pushes the top Hessian eigenvalue (sharpness) up to 2/step size, then keeps it hovering just above that value. Loss rises and falls within short windows but keeps falling over long ones. Shown on CIFAR-10 subsets and standard CNN, VGG and ResNet models. Results are author-reported.Paper · 2021-02-24R2LIVE: A Robust, Real-time, LiDAR-Inertial-Visual tightly-coupled state Estimator and mappingR2LIVE fuses LiDAR, IMU and camera measurements tightly. An error-state iterated Kalman filter runs odometry at IMU rate, and a sliding-window factor graph refines keyframe poses and visual landmarks. Author-reported: on a 300 m D-GPS sequence the median relative pose error is 0.27° and 0.21%, against 0.41° and 0.34% for FAST-LIO.Paper · 2021-02-24Zero-Shot Text-to-Image GenerationA 12-billion-parameter decoder-only transformer models caption text tokens and image tokens as one stream. A discrete VAE maps each 256x256 image to a 32x32 grid of 8192-way tokens. Samples are reranked by a contrastive model. Zero-shot on MS-COCO, human raters preferred its realism over DF-GAN 90% of the time (author-reported).Paper · 2021-02-20Concealed Object DetectionSINet-V2 finds camouflaged objects in two stages, as a predator would: a neighbor connection decoder locates a coarse region, then three group-reversal attention blocks refine it. Also introduces COD10K, 10,000 images with rich annotations. Author-reported: S-measure 0.815 on COD10K test, 0.820 on CAMO, 0.888 on CHAMELEON, beating 12 baselines.Paper · 2021-02-18Unbiased Teacher for Semi-Supervised Object DetectionSemi-supervised Faster R-CNN (FPN, ResNet-50). An EMA teacher pseudo-labels weakly augmented images and the student trains on strong views with RPN and ROI classification losses. Focal loss on the ROI classifier and the EMA teacher counter pseudo-label class bias. Author-reported: 20.75 COCO mAP with 1% labels (STAC 13.97).Paper · 2021-02-17Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual ConceptsCC12M: 12.4M web image-alt-text pairs made by relaxing the CC3M filters, about 4x more data with more long-tail words; captions are less precise (76.6% vs 90.3% for CC3M, from a 100-sample check). A captioner pre-trained on it and fine-tuned on COCO reaches nocaps val CIDEr 87.4 (54.7 without pre-training; 90.2 with CC3M added), author-reported.Paper · 2021-02-15HDMI: High-order Deep Multiplex InfomaxHDMI learns node embeddings on multiplex networks without labels. It trains a one-layer GCN per relation layer by maximising high-order mutual information between node embedding, global summary and node attributes, then fuses layers with semantic attention. Author-reported: highest node-classification Macro-F1 on ACM, IMDB, DBLP and Amazon (0.901, 0.650, 0.820, 0.808).Paper · 2021-02-15Prompt Programming for Large Language Models: Beyond the Few-Shot ParadigmPosition paper arguing few-shot examples mostly locate a task GPT-3 already knows rather than teach it. On WMT'14 Fr-En, a 0-shot "French: ... English:" prompt reaches 33.3 BLEU on Curie vs 27.9 for a reproduced 10-shot GPT-3-style prompt. Proposes natural-language prompt techniques: constraining continuations, persona proxies, serialized reasoning and self-generated metaprompts.Paper · 2021-02-12Understanding Self-Supervised Learning Dynamics without Contrastive PairsAnalyses why non-contrastive SSL (BYOL, SimSiam) avoids collapse without negative pairs, using a linear model with predictor, EMA, stop-gradient and weight decay. Proposes DirectPred: set the linear predictor from the eigendecomposition of the input correlation matrix, no gradient training. ImageNet 300 epochs: 72.4% top-1 vs 69.9% for a gradient-trained linear predictor (author-reported).Paper · 2021-02-09Robust Motion In-betweeningGenerates the frames between a few seed frames and a target keyframe with an LSTM generator and two LSGAN critics. Two additive embeddings do the key work: a sinusoidal time-to-arrival code and a target noise that fades near the keyframe. Beats interpolation on Human3.6M walking and on the new LaFAN1 benchmark (author-reported).Paper · 2021-02-08Deep Residual Learning in Spiking Neural NetworksProposes the spike-element-wise (SEW) residual block: the output is g(A, S), where A is the spiking branch and S the block input, and g is ADD, AND or IAND. For ADD and IAND, identity needs only a zero-initialised last BatchNorm, and gradients pass unchanged. Trained directly on ImageNet at T=4, SEW ResNet-152 reaches 69.26% top-1 where Spiking ResNet-152 reaches 10.03%. Author-reported.Paper · 2021-02-08TransUNet: Transformers Make Strong Encoders for Medical Image SegmentationTransUNet puts a ViT-B/16 encoder behind a ResNet-50 stem and decodes it with a cascaded upsampler that receives skip connections from the CNN. On Synapse abdominal CT it reports 77.48% average DSC, against 75.57% for the best CNN baseline (R50-AttnUNet). On ACDC cardiac MRI it reports 89.71% DSC.Paper · 2021-02-03Embodied Intelligence via Learning and EvolutionDERL co-evolves agent morphologies (MuJoCo kinematic trees of limbs) and PPO controllers with asynchronous tournament evolution; each child learns its controller from scratch. Bodies evolved in more complex terrain learn new tasks faster and use less energy, and learning time to a fitness criterion halves within 10 generations (a morphological Baldwin effect). Author-reported.Paper · 2021-01-31MultiRocket: Multiple pooling operators and transformations for fast and effective time series classificationExtends MiniRocket for univariate time series classification: convolves the raw series and its first difference with MiniRocket's 84 fixed kernels, pools each output with four operators (PPV, MPV, MIPV, LSPV), and fits a ridge classifier on about 50,000 features. Author-reported: more accurate than MiniRocket on the 109-dataset UCR benchmark, not significantly less accurate than HIVE-COTE 2.0.Paper · 2021-01-30Learning From How Humans CorrectFinds likely label noise in an industrial text-classification set with a BERT model, has humans relabel about 300,000 disputed examples, and feeds the first model's predicted label as an extra input to a second BERT model trained on the corrected labels. Test accuracy rises from 83.3% to 92.5% on one private dataset (author-reported; no code).Paper · 2021-01-28Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series ForecastingTimeGrad forecasts multivariate series one step at a time: an LSTM summarises the past and a conditional diffusion model samples each next value from that summary, using 100 denoising steps and 100 sample paths. Author-reported CRPS_sum is lowest on five of six benchmarks; it is not best on Exchange.Paper · 2021-01-28BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG dataBENDR adapts wav2vec 2.0 to EEG: strided 1D convolutions turn raw signals into a short sequence that an 8-layer transformer reads. Pre-training on unlabelled TUEG windows asks the model to pick the true masked vector out of 20 distractors. Fine-tuned on five public tasks, the best author-reported means are 86.7 balanced accuracy (MMI), 42.6% accuracy (BCI IV 2a) and 0.72 AUROC (P300).Paper · 2021-01-27Graph Neural Network for Traffic Forecasting: A SurveySurvey of graph neural networks (GNNs) for traffic forecasting, covering 212 papers from 2018 to 2020. It sorts problems (flow, speed, demand), traffic graphs (road, region and station levels), adjacency types and GNN families, lists open datasets and code, and copies a benchmark table from the surveyed papers. It runs no experiments of its own.Paper · 2021-01-25A Review of Graph Neural Networks and Their Applications in Power SystemsA review of graph neural networks (GCN, GRNN, GAT, generative and spatio-temporal GNNs, hybrids) and their uses in power systems: fault diagnosis, time-series forecasting, power flow and data generation. It runs no experiments of its own; quoted numbers come from cited works.Paper · 2021-01-22A Review on Deep Learning in UAV Remote SensingLiterature review of deep learning on UAV imagery. The authors screened 232 articles and proceedings (2016 to 2021) from Web of Science and Google Scholar, plus 18 reviews, and coded them by network, task, sensor and context. Reported shares: CNNs 91.2%, object detection 53.9%, RGB sensors 52.4%. No model is proposed; counts are the authors' own tallies.Paper · 2021-01-21AI Choreographer: Music Conditioned 3D Dance Generation with AIST++FACT, a full-attention cross-modal transformer that writes 3D dance motion from music and a 2-second seed. Motion and audio go through separate transformers, a 12-layer cross-modal transformer predicts 20 future frames, and rollouts run auto-regressively. Also introduces AIST++, 1408 3D dance sequences. Author-reported FID_k 35.35 against 69.18 to 86.43 for three baselines.Paper · 2021-01-19Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the EdgeClustered federated learning for COVID-19 detection from chest X-ray and lung ultrasound images. Each imaging modality forms a cluster of clients that train a VGG16-based binary classifier locally, and the server averages their weights. Author-reported: F1 comparable to per-modality federated models, and 16% and 11% higher overall F1 than conventional federated learning on X-ray and ultrasound.Paper · 2021-01-18HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPSHarDNet-MSEG segments polyps in colonoscopy images with a HarDNet68 encoder and a small decoder that refines three feature levels with RFB blocks and dense aggregation. It reports 0.904 mean Dice on Kvasir-SEG at 86.7 FPS on an RTX 2080 Ti, and 0.912 under the PraNet protocol.Paper · 2021-01-18Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene GraphsKimera builds a 3D Dynamic Scene Graph, a five-layer map of mesh, objects, humans, places, rooms and building, from stereo or RGB-D images and IMU data. The mesh and pose are built online; the upper layers are parsed after the run. Author-reported: visual-inertial odometry accuracy competitive with other VIO systems on EuRoC.Paper · 2021-01-17Deep Parametric Continuous Convolutional Neural NetworksA learnable convolution whose kernel is an MLP over relative 3D offsets, so it works on unevenly spaced points and can output at any location. Used in deep residual nets for indoor and lidar semantic segmentation and lidar motion estimation. Author-reported: 58.27 mIOU indoor (SEGCloud 48.92), 58.06 mIOU driving with 3D-FCN+PCCN, 7.810 cm EPE for lidar flow (3D-FCN 8.161).Paper · 2021-01-17End-to-end Interpretable Neural Motion PlannerA single CNN takes LiDAR sweeps and an HD map and outputs 3D vehicle detections, their future motion and a space-time cost volume. The planner samples clothoid-based trajectories and picks the one with the lowest summed cost. Trained with detection, forecasting and a max-margin planning loss. Author-reported: lowest future collision rate among compared planners on a private North American dataset.Paper · 2021-01-17TrafficSim: Learning to Simulate Realistic Multi-Agent BehaviorsTrafficSim learns a joint policy for all traffic actors from human driving logs. An implicit latent variable model samples socially consistent plans for every actor at once. Training unrolls the policy through a differentiable simulator, with an imitation loss that fades over time plus a collision penalty. On ATG4D it has the lowest collision rate (0.50%) and reconstruction errors.Paper · 2021-01-16Free Lunch for Few-shot Learning: Distribution CalibrationFew-shot classifier: each support feature's Gaussian is averaged from its 2 nearest base classes (after a sqrt Tukey transform, plus a constant alpha). It draws 750 features per class and trains a logistic regression on them plus the support set. Frozen features, no new parameters. Author-reported 5-way 1/5-shot: miniImageNet 68.57/82.88%, CUB 79.56/90.67%.Paper · 2021-01-14Federated Learning: Opportunities and ChallengesSurvey of federated learning, where devices or organisations train a shared model without sharing raw data. Covers the four-step training loop, FL types, applications, communication and heterogeneity bottlenecks, privacy and poisoning attacks with defences, and recent work. Reports no experiments; ends with an open question on Byzantine-tolerant, private FL.Paper · 2021-01-14Persistent Anti-Muslim Bias in Large Language ModelsProbes GPT-3 (davinci) with completions, analogies and captions. Prompts about Muslims yield violence-related words far more often than prompts about other groups: "Two Muslims walked into a" gives violent completions 66% of the time; "Muslim" is analogized to "terrorist" in 23% of runs. Positive adjectives before the prompt cut violence to 20%. Author-reported.Paper · 2021-01-13Unlearnable Examples: Making Personal Data UnexploitableAdds small (L-inf 8/255) error-minimizing noise to training images, found by a min-min bi-level loop with PGD on a ResNet-18 source model, so models trained on them learn little. Class-wise noise drops CIFAR-10 clean accuracy to 13-17% across four models; sample-wise noise to 19-29%. Author-reported; protection fades when only part of the data is perturbed.Paper · 2021-01-11Adversary Instantiation: Lower Bounds for Differentially Private Machine LearningMeasures how private DP-SGD really is by playing the differential-privacy adversary: a crafter builds two neighbouring datasets, a distinguisher guesses which trained the model, and Clopper-Pearson bounds turn the error rates into an empirical epsilon. With full attacker powers (poisoned data, gradient access) the bound nearly matches the proven epsilon; with black-box access it is far lower.Paper · 2021-01-07Knowledge Distillation in Iterative Generative Models for Improved Sampling SpeedDistills a multi-step DDIM teacher (built on pretrained DDPM or DDIM networks) into one network evaluation. The student starts from the teacher's weights and regresses the teacher's final image from the same initial noise, with no adversarial loss. Author-reported CIFAR-10 FID is 9.36 at one step, against 4.16 for the 100-step teacher.Paper · 2021-01-07TrackFormer: Multi-Object Tracking with TransformersTrackFormer tracks several objects with a DETR-style encoder-decoder: each object's decoder embedding is carried into the next frame as a track query, while learned object queries detect newcomers. Trained on frame pairs with false-positive and false-negative track augmentations. Author-reported: 74.1 MOTA and 68.0 IDF1 on the MOT17 private test set, and 54.9 sMOTSA on MOTS20 test.Paper · 2021-01-06Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning StrategiesStrategyQA: 2,780 yes/no questions whose reasoning steps are implicit and must be inferred from a strategy. Crowd workers write them from Wikipedia terms, solver models filter them, and each has a step decomposition plus Wikipedia evidence per step. Author-reported: a RoBERTa baseline reaches 63.6% accuracy without context and 72.0% with gold paragraphs and decompositions; humans reach 87%.Paper · 2021-01-04Transformers in Vision: A SurveySurvey of Transformers in computer vision. Groups work by self-attention design (CNN add-ons, uniform-scale and multi-scale ViTs, hybrids, self-supervised ViTs) and by task. Notes that ViTs need large pre-training data and that attention cost grows quadratically with the number of tokens. Reports results compiled from the cited papers.Paper · 2020-12-31Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate DetectionBuilds an online hate-speech dataset in four rounds: annotators write posts that fool the current RoBERTa-base target model, half of rounds 2 to 4 are label-flipping perturbations, and each new target is trained on earlier rounds with upsampling. Later targets have lower in-loop error (54.7% to 27.7%), higher test macro F1 and better HateCheck accuracy (60% to 95%). Results are author-reported.Paper · 2020-12-31Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with TransformersSETR replaces the downsampling CNN encoder of segmentation models with a plain ViT: 16x16 patches, no resolution reduction, global self-attention in every layer, then one of three light decoders (Naive, PUP progressive upsampling, MLA multi-level aggregation). Author-reported: 50.28% mIoU on ADE20K val and 55.83% on Pascal Context (SETR-MLA, T-Large, multi-scale).Paper · 2020-12-31The Pile: An 800GB Dataset of Diverse Text for Language ModelingThe Pile: an 825 GiB English corpus of 22 weighted sources (academic, code, legal, web, books, dialogue), built by per-source processing, holdout and exact-document dedup, plus documentation. GPT-2/GPT-3 perplexity is reported in bits per UTF-8 byte. 1.3B-parameter models trained on it beat CC-100 and raw Common Crawl on Pile components and WikiText (author-reported).Paper · 2020-12-27ARBERT & MARBERT: Deep Bidirectional Transformers for ArabicTwo BERT-Base encoders for Arabic: ARBERT, pre-trained on 61 GB of mostly formal Arabic, and MARBERT, pre-trained on 1B Arabic tweets with no next-sentence objective. Both use a 100K WordPiece vocabulary. Also introduces ARLUE, 42 datasets in six task clusters. MARBERT-v2 scores 77.40 ARLUE (author-reported), above XLM-R Large at 76.55.Paper · 2020-12-25Logic Tensor NetworksLogic Tensor Networks (LTN) ground first-order logic in real-valued tensors (Real Logic), so a knowledge base's satisfiability becomes a differentiable training objective. It covers classification, clustering, regression, embedding learning and refutation-based reasoning. Author-reported: 98.04% test accuracy on single-digit MNIST addition with 30k training examples, versus 97.20% for DeepProbLog.Paper · 2020-12-23Methods to integrate multinormals and compute classification measuresComputes the probability that a normal vector falls in any domain: a generalized chi-square method for quadratic boundaries, a ray-trace method for any shape. Adds Bayes-optimal error rates, a discriminability index d'_b and tests of the normal model. Author-reported: a Matlab toolbox matches Monte Carlo where sampling works, and stays accurate to d' about 75 where sampling fails beyond about 10.Paper · 2020-12-22Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-TuningFine-tunes a pretrained language model by training only d coordinates of a fixed random projection (Fastfood); SAID adds per-layer scales. RoBERTa-Large reaches 90% of full MRPC fine-tuning with about 200 trainable parameters. Pre-training lowers this intrinsic dimension, and lower values track better generalization, with a compression-based bound (author-reported).Paper · 2020-12-19Multi-Decoder Attention Model with Embedding Glimpse for Solving Vehicle Routing ProblemsTrains several Transformer-based attention decoders with REINFORCE and a KL diversity term to build routing tours, adds an Embedding Glimpse layer that re-embeds unvisited nodes every few steps, and uses a beam search with one beam per decoder. Author-reported: 0.38% gap on TSP100 and 2.23% on CVRP100 with 50-wide beams.Paper · 2020-12-18Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and DefensesSurvey of dataset security for machine learning. It sorts training-only data poisoning, backdoor (trojan) attacks and their defenses into one taxonomy, covers federated-learning threats and lists open problems. It is a survey with no experiments of its own, so it reports no benchmark numbers.Paper · 2020-12-18TDN: Temporal Difference Networks for Efficient Action RecognitionAdds two temporal difference modules to a 2D ResNet: a short-term one that uses RGB differences of neighbouring frames in early stages, and a long-term one that uses differences between sampled segments as attention in later stages. Author-reported: 52.3% top-1 on Something-Something V1 with 8 frames of ResNet-50, and state of the art on SSv1 and SSv2 among RGB-only models.Paper · 2020-12-17Reconstructing Hand-Object Interactions in the WildReconstructs the 3D hand and a known-model object from one in-the-wild RGB image, without 3D supervision. It fits MANO to 2D keypoints, fits the object pose by differentiable mask and depth rendering, optimises both jointly with distance and collision terms, then refines the hand with a learned contact prior. Author-reported: HO3D hand error 9.7 mm vs 14.7 mm for the baseline.Paper · 2020-12-17Transformer Interpretability Beyond Attention VisualizationComputes class-specific relevance maps for Transformer classifiers. Relevance is propagated back through attention, skips and matrix products with LRP-style rules; each block's gradient-weighted relevance plus identity is multiplied across blocks and the [CLS] row is read out. Author-reported: best on ImageNet perturbation and segmentation tests against rollout, attention, GradCAM and LRP.Paper · 2020-12-16Learning Self-Consistency for Deepfake DetectionDetects face forgeries by checking whether the patches of an image still share one source. A ResNet-34 predicts a 4D pairwise patch-consistency volume (BCE loss) plus a real/fake head. I2G makes training data by stitching real face crops with elastically deformed masks. Author-reported AUC: in-dataset average 96.45 to 98.05 (FF++, CD2, DFDC-P); cross-dataset average 86.03 to 92.18.Paper · 2020-12-15Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object DetectionEqualization Loss v2 reweights the per-class sigmoid loss of an end-to-end detector. Each class tracks its accumulated positive-to-negative gradient ratio, and a sigmoid map of that ratio up-weights positives and down-weights negatives. Author-reported on LVIS v1.0 with a ResNet-50 Mask R-CNN (1x): 23.7 AP against 18.6 for EQL and 16.1 for softmax, with no fine-tuning stage.Paper · 2020-12-15Spectral Methods for Data Science: A Statistical PerspectiveMonograph on spectral methods: estimate a hidden matrix's top eigen- or singular vectors from a matrix built from data. Gives sin-Θ perturbation bounds, leave-one-out entrywise (ℓ∞, ℓ2,∞) guarantees, and spectral estimators for denoising, PCA, SBM clustering, ranking, phase retrieval and matrix and tensor completion. Results are proven bounds, not benchmarks.Paper · 2020-12-14HR-Depth: High Resolution Self-Supervised Monocular Depth EstimationTrains a monocular depth network from video frames only. Its U-Net decoder gets dense skip connections and a feature-fusion SE block, aimed at sharp edges at high input resolution. A MobileNetV3 variant, distilled from a larger model, beats Monodepth2 at high resolution with 3.1M parameters. Best KITTI Abs Rel 0.101 with stereo supervision (author-reported).Paper · 2020-12-13Simple Copy-Paste is a Strong Data Augmentation Method for Instance SegmentationCopy-Paste augmentation for instance segmentation: scale-jitter two random training images, paste a random subset of one image's object masks onto the other, and fix up occluded masks and boxes. No context modelling. Combined with large scale jittering and self-training, Cascade Eff-B7 NAS-FPN reaches 57.3 box AP / 49.1 mask AP on COCO test-dev; +6.1 mask AP_r on LVIS.Paper · 2020-12-11TabTransformer: Tabular Data Modeling Using Contextual EmbeddingsTabTransformer embeds each categorical column, runs Transformer layers over those embeddings so each becomes context-dependent, then feeds them with numeric features to an MLP. On 15 binary datasets it averages 82.8% AUC, 1.0 point above an MLP and level with GBDT (author-reported). For semi-supervised use, it pretrains the Transformer on unlabeled rows with masking or replaced-value detection.Paper · 2020-12-10An IoT Framework for Heart Disease Prediction based on MDCNN ClassifierProposes an IoT heart-disease classifier. Features are picked by a modified cuttlefish search (MCFA), and a fully connected network (MDCNN) has its weights set by elephant-herd optimization (AEHO). Author-reported: 98.2% accuracy on a 4000-record Framingham set, against 88.3% for logistic regression and 83.8% for a plain deep network. No official code.Paper · 2020-12-10Neurosymbolic AI: The 3rd WavePosition paper on neurosymbolic AI: combining neural learning with symbolic knowledge and logical reasoning for trust, explainability and extrapolation. It maps 20 years of neural-symbolic work onto Kautz's six-type taxonomy, argues that localist versus distributed representation is the central design choice, and lists three open challenges. It has no experiments and reports no benchmark numbers.Paper · 2020-12-09On the Binding Problem in Artificial Neural NetworksPosition survey arguing that neural networks fall short of human-level generalization because they cannot dynamically bind distributed information into symbol-like object representations. Splits the binding problem into representation, segregation and composition, reviews mechanisms for each (slots, attention, GNNs, relational frames) and lists open problems. No experiments or equations.Paper · 2020-12-08Ditto: Fair and Robust Federated Learning Through PersonalizationDitto trains one personal model per device, pulled toward the shared global model by an L2 term weighted by lambda. The global model uses any federated solver (FedAvg in the experiments). Authors report about 6% higher benign accuracy than the strongest robust baseline under attack, and about 10% lower accuracy variance than a fair baseline (TERM).Paper · 2020-12-08Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMap, and PaCMAP for Data VisualizationPaCMAP is a dimension reduction method for visualisation. It attracts nearest neighbours, attracts a few mid-near pairs early on to fix global layout, then drops them and refines local structure, while repelling random far points. Reported to preserve local and global structure without relying on initialisation, and to run faster than t-SNE, LargeVis, UMAP and TriMap.Paper · 2020-12-07Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene CompletionJS3C-Net segments one LiDAR sweep with a sparse-voxel U-Net trained jointly with a scene-completion decoder. Its targets are voxels built from 70 merged sweeps. A point-voxel interaction module links the two tasks, and the completion branch is dropped at inference. Author-reported: 66.0 mIoU on SemanticKITTI and 60.2 on SemanticPOSS.Paper · 2020-12-03BasicVSR: The Search for Essential Components in Video Super-Resolution and BeyondKeeps video super-resolution simple: bidirectional recurrent propagation, optical-flow warping of features, residual blocks and pixel-shuffle upsampling. IconVSR adds keyframe information-refill and coupled propagation. Author-reported: BasicVSR gets 31.42 dB PSNR on REDS4 (BI, x4) with 6.3M parameters and 63 ms, against EDVR's 31.09 dB with 20.6M and 378 ms.Paper · 2020-12-03GottBERT: a pure German Language ModelGottBERT is a German-only RoBERTa (base and large) pretrained with fairseq on the German part of OSCAR (145GB), plus a 121GB version filtered with heuristic rules and a one-class SVM. The authors report that base models lead on 4 of 6 tasks (NLI, two NER, three classification) against German BERT models and two multilingual ones; filtering had no clear effect.Paper · 2020-12-03pixelNeRF: Neural Radiance Fields from One or Few ImagespixelNeRF conditions a NeRF on pixel-aligned ResNet34 features of one or more posed input images, predicts colour and density in each input camera's frame, and pools views in the middle of the MLP. Trained only on multi-view 2D images, it renders novel views feed-forward with no per-scene optimisation. Author-reported: 23.72 PSNR (1-view chairs) and 26.80 mean PSNR over 13 ShapeNet categories.Paper · 2020-12-02Few-Shot Classification with Feature Map Reconstruction NetworksFRN classifies a few-shot query by how well each class's support feature maps reconstruct the query's feature map. The reconstruction is a closed-form ridge regression with two learned scalars for regularisation and recalibration. Author-reported: 83.16% (1-shot) and 92.59% (5-shot) on cropped CUB with ResNet-12; 82.83% 5-shot on mini-ImageNet.Paper · 2020-11-30End-to-End Video Instance Segmentation with TransformersVisTR casts video instance segmentation as direct sequence prediction: a CNN and a Transformer encoder-decoder over a 36-frame clip emit 10 instance sequences of masks in one pass, matched and supervised as whole sequences, with no tracking heuristics. Author-reported: 40.1 mask AP on YouTube-VIS val with ResNet-101, 27.7 FPS with data loading.Paper · 2020-11-30Inductive Biases for Deep Learning of Higher-Level CognitionA position paper arguing that deep learning needs extra inductive biases taken from higher-level, conscious cognition to generalise out of distribution and systematically. It proposes verbalisable causal variables, sparse and local changes, a sparse factor graph over them, reusable generic knowledge and short causal chains. It gives no algorithm, experiment or numbers.Paper · 2020-11-28AdaBins: Depth Estimation using Adaptive BinsMonocular depth network whose head predicts per-image bin widths over the depth range with a small transformer, plus per-pixel range-attention maps; depth is the softmax-weighted average of bin centres. Author-reported: NYU-Depth-v2 delta1 0.903 and KITTI RMS 2.360, presented as state of the art.Paper · 2020-11-27General Multi-label Image Classification with TransformersC-Tran is a Transformer over image grid features and one token per label, each tagged unknown, negative or positive. Label mask training hides random labels and predicts them, so known labels can be given at test time. Author-reported: 85.1 mAP on COCO-80 and 38.4 on VG-500, without label graphs, with gains from partial or extra labels.Paper · 2020-11-25PREDATOR: Registration of 3D Point Clouds with Low OverlapRegisters two 3D point clouds that share little surface area. An overlap-attention block exchanges information between the two encodings early, and predicts per-point overlap and matchability scores that bias keypoint sampling. Author-reported: 59.8% registration recall on 3DLoMatch (10-30% overlap) vs 40.1% for FCGF; 89.0% on 3DMatch.Paper · 2020-11-23Better Aggregation in Test-Time AugmentationTest-time augmentation averages predictions over transformed copies of an image, but averaging can flip correct predictions to wrong ones. The method learns non-negative weights per augmentation (AugTTA) or per augmentation and class (ClassTTA) on a held-out split. Author-reported: gains over mean, max and GPS on most ImageNet and Flowers-102 models, not uniformly.Paper · 2020-11-23PLOP: Learning without Forgetting for Continual Semantic SegmentationPLOP adds new classes to a semantic segmentation network without old training images. It combines Local POD, a multi-scale feature distillation that pools widths and heights over grid regions, with entropy-thresholded pseudo-labels that give old-class labels to confident background pixels. Author-reported Pascal VOC 2012 overlapped 15-1 mIoU: 54.64 vs 29.29 for the authors' MiB re-implementation.Paper · 2020-11-22Interpreting Super-Resolution Networks with Local Attribution MapsLAM explains a super-resolution network by attributing one output patch to the input pixels that drive it. It is integrated gradients with a blurred low-resolution baseline and a path that progressively sharpens the blur. Across 48 networks, the Gini-based diffusion index of each map correlates with PSNR (Pearson and Spearman above 0.8). Results are author-reported.Paper · 2020-11-21Rethinking Transformer-based Set Prediction for Object DetectionTSP-FCOS and TSP-RCNN replace DETR's decoder with a 6-layer transformer encoder over 700 features of interest from FCOS or an RPN, trained with a restricted Hungarian set loss. Reported COCO val AP at R-50, 36 epochs: 43.1 and 43.8; DETR reaches 42.0 after 500 epochs.Paper · 2020-11-20Neural Scene Graphs for Dynamic ScenesDecomposes a dynamic driving scene into a graph: one static background radiance field plus one field per tracked object, with objects of a class sharing weights through a per-object latent code. Tracking data place the objects, so they can be moved, removed or recombined and rendered from new views. On three KITTI scenes it reports PSNR 26.66 vs 23.34 for NeRF (author-reported).Paper · 2020-11-19Node Similarity Preserving Graph Convolutional NetworksSimP-GCN extends a two-layer GCN. Each node mixes the graph with a cosine kNN graph built from its features, using a learned score, and adds learnable self-loop weights. A self-supervised regression on sampled pairs' feature similarity is added to the loss. Author-reported: it beats GCN on seven benchmarks, e.g. Cornell 84.05% vs 52.70%.Paper · 2020-11-18Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?Independent PPO (IPPO) gives each agent its own PPO actor and local critic, trained on a shared team reward with shared weights. On SMAC it matches or beats MAPPO, QMIX and other joint-value methods on several hard maps, with mild per-map tuning. Ablations point to policy clipping as the key ingredient. All results are author-reported.Paper · 2020-11-17Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin NetworksModels offloading of stochastic IIoT tasks in a digital-twin network as a long-term energy-efficiency problem. Lyapunov drift-plus-penalty turns each slot into a deterministic problem, solved with asynchronous actor-critic (AAC). Author-reported: AAC lowers simulated system cost versus two resource-allocation variants and DQN; no numeric values in the text.Paper · 2020-11-17Low-latency Federated Learning and Blockchain for Edge Association in Digital Twin empowered 6G NetworksDigital twins of users sit on base stations, which run federated learning and log local models on a permissioned blockchain (DPoS). A multi-agent DDPG, one agent per base station, picks twin association, batch sizes and bandwidth to cut training time at a target accuracy. CIFAR-10, 100 users; results only in figures, lower time cost than two simple baselines.Paper · 2020-11-17RELLIS-3D Dataset: Data, Benchmarks and AnalysisRELLIS-3D is an off-road multimodal dataset from a ground robot on a university campus: 6,235 pixel-labelled camera images and 13,556 point-labelled LiDAR scans in 20 classes, with raw sensor streams. Author-reported baselines reach 51.6 to 52.9 mIoU on images (HRNet+OCR, Gated-SCNN) and 20.0 to 43.1 on point clouds (KPConv, SalsaNext), well below urban-data results.Paper · 2020-11-16Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge BasesBERT-based seq2seq parser for knowledge base QA. It encodes the question and schema items together, then decodes a logical form over a 2-hop-pruned vocabulary or ranks enumerated 2-hop candidates. Also releases GrailQA (64,331 questions) for i.i.d., compositional and zero-shot tests. Author-reported: ranking with BERT reaches 58.0 F1 overall and 33.8 F1 zero-shot without BERT.Paper · 2020-11-15Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep LearningBenchmarks detectors (EfficientDet-D0, Faster R-CNN, RetinaNet, YOLOv3, YOLOv4) and segmenters (UNet, ResUNet, ResUNet++, FCN8, HRNet, DoubleUNet, PSPNet, DeepLabv3+) on Kvasir-SEG. Proposes ColonSegNet, a ~5M-parameter residual encoder-decoder with squeeze-and-excitation. Author-reported: Dice 0.8206 at 182.38 FPS; detection AP 0.8000 at 180 FPS.Paper · 2020-11-13Learning to Drop: Robust Graph Neural Network via Topological DenoisingPTDNet trains a small per-layer network with the GNN that scores each edge and drops likely task-irrelevant ones. It uses a hard-concrete relaxation, a penalty on the expected number of kept edges and a nuclear-norm (low-rank) penalty. Author-reported: GCN on Cora 0.828 vs 0.811 for the basic model, and larger gains with injected noisy edges.Paper · 2020-11-12Deep Partial Multi-View LearningCPM-Nets learns one shared latent code per sample from whatever views the sample has. Per-view networks rebuild the observed views from the code, and a clustering-style class loss gives the codes structure. A GAN variant (CPM-GAN) imputes missing views. Author-reported gains over baselines, mostly shown in figures.Paper · 2020-11-11Unsupervised Learning of Dense Visual RepresentationsLearns per-pixel features without labels: two augmented views are matched pixel to pixel via their known geometry, and an InfoNCE loss pulls matches together against a MoCo queue of 65,536 negatives. Author-reported: with frozen features it beats MoCo and ImageNet-supervised ResNet-50 on VOC mIoU and NYU depth RMSE, and beats MoCo on all nine COCO Mask R-CNN metrics.Paper · 2020-11-10CenterFusion: Center-based Radar and Camera Fusion for 3D Object DetectionCenterFusion adds radar to a camera-only center-point 3D detector. Each object's radar points are picked by a frustum-based association, and their depth and velocity are drawn as extra image-shaped channels for second-stage heads that refine depth, rotation and velocity. On nuScenes validation with a DLA backbone, NDS rises from 0.328 to 0.453 (author-reported).Paper · 2020-11-10Debugging Tests for Model ExplanationsTests whether feature attributions (Gradient, SmoothGrad, Integrated Gradients, LRP and others) reveal four planted bugs: spurious backgrounds, mislabelled data, re-initialised weights and out-of-distribution inputs. Attributions show the spurious background but not wrong labels; modified-backprop methods ignore re-initialised top layers. A 54-person study found users relied on predictions.Paper · 2020-11-09Parameterized Explainer for Graph Neural NetworkPGExplainer trains one small MLP that scores the edges of a frozen GNN and explains many instances at once, including unseen ones, without per-instance optimisation. Edges are relaxed with binary concrete samples and trained to keep the GNN's prediction. Author-reported explanation AUC is 0.87-0.99 on five synthetic datasets and MUTAG, and inference is 12x-108x faster than GNNExplainer.Paper · 2020-11-04Learning and Evaluating Representations for Deep One-class ClassificationTwo-stage deep one-class classification: self-supervised features learned from inlier images only, then a kernel density estimator or RBF one-class SVM on them. Adds distribution-augmented contrastive learning, which treats rotated training copies as extra distributions. Author-reported mean AUC 89.9 over five benchmark settings with OC-SVM, without test-time augmentation.Paper · 2020-11-02Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning StepsBuilds 2WikiMultiHopQA, about 193k multi-hop questions from Wikipedia summaries and Wikidata triples. Each question carries evidence triples that spell out its reasoning path. A HotpotQA-style baseline reaches 5.41 joint F1 on the test split (author-reported), so the dataset is hard for it.Paper · 2020-10-31A Survey on Contrastive Self-supervised LearningSurvey of contrastive self-supervised learning for vision and NLP. It covers pretext tasks, four ways to store negatives (end-to-end batch, memory bank, momentum queue, clustering), the cosine-based NCE and InfoNCE losses, and ImageNet, Places, VOC and Kinetics numbers copied from the original papers. Not a method; no code.Paper · 2020-10-27Graph Contrastive Learning with Adaptive AugmentationGCA learns node embeddings of a graph without labels. It contrasts two randomly corrupted views of the graph, dropping edges and masking feature dimensions with probabilities set by node centrality (degree, eigenvector or PageRank), so important structure survives. Author-reported node classification accuracy beats other unsupervised graph methods on five benchmarks.Paper · 2020-10-22An Analysis of Simple Data Augmentation for Named Entity RecognitionFour simple token-level augmentations for NER (label-wise token replacement, WordNet synonym replacement, mention replacement, shuffle within segments), tested with an LSTM and a SciBERT tagger on two domain datasets. Author-reported: gains are largest on small training sets; applying all four averages +3.3 F1 (LSTM) and +1.8 F1 (SciBERT) across eight columns.Paper · 2020-10-16Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring TasksAugmented SBERT labels extra sentence pairs with a slow, accurate BERT cross-encoder and trains a fast bi-encoder on gold plus these soft-labelled pairs. Which pairs are chosen decides the gain: BM25 or KDE sampling helps, random pairs do not. Author-reported: up to 6 points in-domain, up to 37 points on domain adaptation.Paper · 2020-10-10Contrastive Representation Learning: A Framework and ReviewReview that casts contrastive learning as one framework: a similarity distribution that samples positive and negative keys for a query, encoders (end-to-end, online-offline, pre-trained), transform heads (projection, contextualisation, quantisation) and a loss (margin, NCE-based, MI-based). Classifies about 50 methods across language, vision, audio, graphs and multimodal data. No experiments.Paper · 2020-10-06COSMIC: COmmonSense knowledge for eMotion Identification in ConversationsCOSMIC labels the emotion of each utterance in a conversation. It fine-tunes RoBERTa-Large for utterance features, adds five COMET commonsense vectors per utterance, and tracks speaker, listener, intent and emotion states with GRUs. Authors report 65.28 weighted F1 on IEMOCAP and new bests on DailyDialog, MELD and EmoryNLP.Paper · 2020-09-29INSPIRED: Toward Sociable Recommendation Dialog SystemsIntroduces INSPIRED, 1,001 human-human movie-recommendation dialogs labelled per utterance with sociable strategies, and a GPT-2 dialog model that prepends the strategy label to each recommender turn. Author-reported: on the test split the strategy model has perplexity 8.93 vs 9.28 and BLEU-4 6.63 vs 5.11 for the baseline; humans prefer it on all five aspects.Paper · 2020-09-17Face Mask Detection using Transfer Learning of InceptionV3Fine-tunes an ImageNet-pretrained InceptionV3 with a 5x5 average pool, a 128-unit ReLU dense layer, dropout 0.5 and a 2-way softmax to classify whether a person wears a mask. Trained on augmented images of the Simulated Masked Face Dataset (SMFD). Author-reported: 99.92% training and 100% testing accuracy.Paper · 2020-09-01PyCSP3: Modeling Combinatorial Constrained Problems in PythonPyCSP3 is a Python library for declaring constraint satisfaction and optimization models. A model and its JSON data compile into an XCSP3 XML instance, which solvers such as ACE and Choco read. The document is the library's user guide: variables, objectives, about 27 constraint forms, solver piloting and pitfalls. It reports no benchmark results.Paper · 2020-08-30A Survey of Deep Active LearningSurvey of deep active learning (DeepAL) with references up to November 2020. Frames three obstacles to combining AL with deep nets (overconfident softmax uncertainty, too few labels, split pipelines) and sorts work into batch query strategies (uncertainty, hybrid, Bayesian, density/core-set, learned), label expansion (pseudo-labels, GANs, semi-supervised), frameworks and stopping. No experiments.Paper · 2020-08-07Review of Swarm Intelligence-based Feature Selection MethodsSurveys feature selection with 11 swarm-intelligence search methods (PSO, ACO, ABC, DE, GSA, FA, BA, COA, GWO, WOA, SSA) and categorises filter, wrapper, embedded and hybrid selection. Runs one wrapper per method and four filters on six UCI datasets with SVM, Naive Bayes and AdaBoost. PSO has the best wrapper average on SVM and AdaBoost; ACO leads filters. Author-run.Paper · 2020-08-06StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing FlowsTrains a conditional continuous normalizing flow between StyleGAN's W latent and a standard normal, conditioned on 17 face attributes. Sampling runs the flow with target attributes; editing re-encodes an image with its source attributes and decodes with target ones. Author-reported: lower FID and higher identity scores than latent-arithmetic and linear baselines on StyleGAN2 faces.Paper · 2020-07-31The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiesSurvey of explainable AI for health care. Defines explainability as interpretability plus fidelity, sorts post-hoc methods into model-based, attribution-based and example-based explanations (global or local), lists which quantitative metrics exist for each property, and gives a decision guide. No experiments; few metrics exist for local clarity and example-based methods.Paper · 2020-07-27FedML: A Research Library and Benchmark for Federated Machine LearningFedML is an open library and benchmark for federated learning research. It separates a worker-oriented message API and topology manager from training, runs on three paradigms (on-device, MPI distributed, single-machine simulation), ships standard algorithms, and fixes non-IID datasets and models. Author-reported: ResNet-56 with FedAvg on CIFAR-10 falls from 93.19 (IID) to 87.12 (non-IID).Paper · 2020-07-26KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social MediaFine-tunes a language-specific BERT-base and feeds its last four hidden layers into a CNN with 160 filters (five widths) and a sigmoid output, for offensive-tweet detection in Arabic, Greek and Turkish. Author-reported macro-F1 on OffensEval 2020 subtask A: 0.897 (Arabic), 0.843 (Greek), 0.814 (Turkish), ranked 4th, 4th and 3rd. Also releases ArabicBERT.Paper · 2020-07-21MovieNet: A Holistic Dataset for Movie UnderstandingMovieNet: 1,100 movies with trailers, photos, scripts, synopses and subtitles, plus manual labels for characters, scene boundaries, actions, places and camera style. Defines nine benchmark tasks with baselines. Author-reported: Cascade R-CNN 95.17 mAP for character detection; MS-LSTM 0.465 AP for scene segmentation.Paper · 2020-07-10The Computational Limits of Deep LearningEstimates the training compute behind deep-learning results from about 1,526 arXiv papers across six benchmarks, then fits benchmark error against compute. The ImageNet fit implies compute grows roughly as performance to the power 12, so halving error needs about 5,000 times more compute. The authors call current trends unsustainable. All numbers are author-reported.Paper · 2020-07-08Auto-Sklearn 2.0: Hands-free AutoML via Meta-LearningAuto-Sklearn 2.0 chains three ideas: a precomputed greedy portfolio warm-starts Bayesian optimization, successive halving allocates budgets, and a random-forest selector picks one of eight policies per dataset from two meta-features. Author-reported on 39 datasets: normalized balanced error 3.58 at 10 min and 2.47 at 60 min, vs 16.21 and 7.17 for Auto-sklearn 1.0.Paper · 2020-07-08General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning ModelsReview of pitfalls when interpreting ML models with model-agnostic methods such as PDP, ALE, permutation importance, SHAP and ICE. Ten areas (generalization, feature dependence, interactions, uncertainty, high dimensions, causality), each with an example, a fix and open issues. Illustrated by simulations with public code. No new method, no benchmark.Paper · 2020-07-08SmaAt-UNet: Precipitation Nowcasting using a Small Attention-UNet ArchitectureSmall U-Net for 30-minute precipitation nowcasting: 12 radar maps in, one predicted map out. CBAM attention on the skip paths and depthwise-separable convolutions cut parameters to about a quarter of U-Net (4.1M vs 17.3M). On the NL-50 test set it ties U-Net on MSE (0.0122) and is slightly better on NMSE; author-reported.Paper · 2020-06-25One Thousand and One Hours: Self-driving Motion Prediction DatasetOpen self-driving dataset of about 1,100 hours of perception output from 20 vehicles on one Palo Alto route, with an HD semantic map and an aerial map. Baselines are a ResNet-50 on bird's-eye-view rasters: motion forecasting error falls as training data grows (ADE 2.47 m at 1% of data without history, 0.77 m at 100% with 1 s history). Planning is closed-loop and reported only as a plot.Paper · 2020-06-22Bayesian Neural Networks: An Introduction and SurveySurvey of Bayesian neural networks: a prior over weights, the posterior, and predictions averaged over it. Covers variational inference, Hamiltonian Monte Carlo, the GP limit, Bayes by Backprop and MC dropout. Two small author experiments: MC dropout vs Bayes by Backprop vs a GP on toy regression, and a Bayesian LeNet on MNIST (98.99% vs 99.92% for vanilla). No hyperparameters, no code.Paper · 2020-06-17Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance AwarenessSNGP makes one deterministic network report uncertainty by spectrally normalizing its hidden weights and replacing the output layer with a random-feature Gaussian process with a closed-form covariance. Author-reported: on CIFAR-10/100 (Wide ResNet-28-10) and CLINC OOS (BERT-Base) it is competitive with a 10-model deep ensemble on accuracy, calibration and OOD detection, at single-model latency.Paper · 2020-06-15Self-supervised Learning: Generative or ContrastiveSurvey of self-supervised representation learning in vision, NLP and graphs. Sorts methods by objective into generative (AR, flow, autoencoder), contrastive (context-instance and instance-instance) and generative-contrastive (adversarial), lists their losses, reviews MI, ELBO and contrastive-loss theory, and names open problems. No experiments of its own.Paper · 2020-06-14Adaptive Universal Generalized PageRank Graph Neural NetworkGPR-GNN learns one weight per propagation step (K=10) of a normalised-adjacency sum, trained jointly with a 2-layer MLP, so the weights can turn negative and act as a high-pass filter on heterophilic graphs. Author-reported: best mean accuracy on 7 of 10 benchmarks, e.g. Texas 92.92 vs 92.26 for MLP.Paper · 2020-06-12Ensemble Distillation for Robust Model Fusion in Federated LearningFedDF fuses federated client models by distilling the averaged logits of all received clients into one server model, trained on unlabeled or generated data, instead of averaging weights. This allows heterogeneous architectures and precisions. Author-reported: on CIFAR-10 (ResNet-8, 20 clients, 40% sampled, 40 local epochs) it reaches 80% top-1 in 20 ± 4 rounds versus 104 ± 52 for FedAvg.Paper · 2020-06-09COVID-ABS: An Agent-Based Model of COVID-19 Epidemic to Simulate Health and Economic Effects of Social Distancing InterventionsAgent-based SEIR model of a closed, Brazil-parameterised society of people, households, businesses, a government and a health system. It runs seven social-distancing scenarios and reports deaths, infection peak and wealth change. Lockdown minimises deaths but hurts business most; masks plus 50% partial isolation is the best option when lockdown is infeasible. Author-reported; no validation.Paper · 2020-06-06GRNet: Gridding Residual Network for Dense Point Cloud CompletionGRNet completes a partial 3D point cloud: it places the points on a 64^3 grid with trilinear weights, runs a residual 3D encoder-decoder, maps the grid back to points, samples grid features at 2,048 coarse points and predicts 8 offsets per point. Author-reported, in x10^-4 Chamfer: 2.723 on ShapeNet (PCN 4.016) and 10.64 on Completion3D (TopNet 14.25).Paper · 2020-06-05Sentiment Analysis Based on Deep Learning: A Comparative StudyCompares a DNN, a CNN and an LSTM (labelled RNN) for binary positive/negative sentiment on eight tweet and review datasets, fed with word embeddings or TF-IDF, under 10-fold cross-validation. Word embeddings beat TF-IDF for CNN and RNN on all eight datasets. The LSTM with embeddings reached 0.828 accuracy on Sentiment140. Author-reported.Paper · 2020-06-02On the Predictive Power of Neural Language Models for Human Real-Time Comprehension BehaviorTests over two dozen language models (5-gram, LSTM, RNNG, Transformer; four BLLIP sizes) by how well their word surprisal explains human reading times on Dundee eye-tracking and two self-paced reading sets. Predictive power is held-out ΔLogLik from a linear regression. Author-reported: it rises as perplexity falls, Transformers and 5-grams beat LSTMs, and syntactic generalization adds nothing.Paper · 2020-06-02SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better RegularizationSaliencyMix cuts a patch centred on the most salient pixel of one training image, pastes it at the same spot on another image and mixes labels by patch area. Author-reported: 2.76% top-1 error on CIFAR-10 and 16.56% on CIFAR-100 for WideResNet-28-10 with traditional augmentation; 21.26% and 20.09% top-1 error on ImageNet for ResNet-50 and ResNet-101.Paper · 2020-05-27CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and PatientsCLOCS pre-trains a 1D CNN on ECG with contrastive learning. Positives are segments that share a patient: adjacent time segments, different leads, or both. Author-reported linear AUC on Chapman rises from 0.738 (SimCLR) to 0.896 at 50% labels, 4 leads, E=128.Paper · 2020-05-26End-to-End Object Detection with TransformersDETR treats object detection as set prediction: a CNN backbone plus transformer encoder-decoder turns N=100 learned object queries into boxes and classes in parallel, trained with a Hungarian-matching loss, with no anchors or NMS. ResNet-50 DETR reaches 42.0 AP on COCO val, on par with a tuned Faster R-CNN-FPN (42.0), better on large and worse on small objects.Paper · 2020-05-25Thermodynamics-based Artificial Neural Networks for constitutive modelingThermodynamics-based neural networks (TANNs) for strain-rate-independent material models. A network predicts a free-energy potential and internal-variable increments; stress and dissipation follow by automatic differentiation, so the first law holds by construction. On 1D and 3D elasto-plastic tests, the authors report better accuracy and generalization than a standard ANN.Paper · 2020-05-17TaBERT: Pretraining for Joint Understanding of Textual and Tabular DataTaBERT is a BERT-initialized encoder pretrained on 26.6M web and Wikipedia tables with nearby text. It reads a question with K sampled table rows, and vertical self-attention shares information across rows. As the encoder of two semantic parsers it beats the previous WikiTableQuestions ensemble (author-reported 52.3% test, best dev run) and is competitive on Spider dev.Paper · 2020-05-15COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on TwitterCT-BERT continues BERT-Large's masked-language-model and next-sentence pretraining on about 22.5M cleaned COVID-19 tweets, then fine-tunes for classification. Across five classification datasets, mean F1 rises from 0.802 to 0.833 (author-reported), with the largest gain on COVID-19 category tweets.Paper · 2020-05-07MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment AnalysisMISA maps each modality (text, video, audio) of an utterance into a shared, modality-invariant subspace trained with a CMD similarity loss and a private, modality-specific subspace trained with difference and reconstruction losses. A one-layer Transformer fuses the six vectors. Author-reported: MOSI MAE 0.783 and MOSEI MAE 0.555 (state of the art), UR-FUNNY Acc-2 70.61 with BERT.Paper · 2020-05-04Monitoring COVID-19 social distancing with person detection and tracking via fine-tuned YOLO v3 and Deepsort techniquesDetects and tracks people in surveillance video to monitor social distancing. A YOLO v3 person detector, fine-tuned on Open Images, is tracked with DeepSORT; people closer than a pixel threshold form groups, and the violation index is people per group. Author-reported: YOLO v3 with DeepSORT gives the best balance of mAP (0.846) and FPS (23).Paper · 2020-05-01Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis ResearchPosition paper, not a survey. It argues that sentiment analysis is far from solved, traces its milestones from rule-based lexicons to transformer encoders, and lists open directions: aspect-level, multimodal, conversational context, sentiment reasoning, domain adaptation, multilingual, sarcasm, sentiment-aware generation and bias. It reports no experiments.Paper · 2020-04-30Fact or Fiction: Verifying Scientific ClaimsIntroduces SciFact, 1,409 expert-written scientific claims checked against 5,183 abstracts with support or refute labels and rationales, and VeriSci, a baseline of TF-IDF retrieval, RoBERTa rationale selection and RoBERTa label prediction. Author-reported open-setting test F1: 39.5 (sentence, selection with label) and 46.5 (abstract, label with rationale).Paper · 2020-04-29Neural Additive Models: Interpretable Machine Learning with Neural NetsNeural Additive Models (NAMs) give each input feature its own small neural net and sum the outputs plus a bias, so the model is a differentiable GAM. ExU hidden units fit sharp shape functions. On four tabular datasets, 5-fold results are close to boosted-tree GAMs (EBMs), author-reported.Paper · 2020-04-28PODNet: Pooled Outputs Distillation for Small-Tasks Incremental LearningPODNet does class-incremental image classification with a 20-images-per-class memory. It distills width- and height-pooled feature maps of every ResNet stage plus the final embedding (POD), and classifies with a multi-proxy cosine classifier (LSC). Reports +12.10 p.p. average incremental accuracy over UCIR on CIFAR100 with 50 one-class steps.Paper · 2020-04-24Deep Global RegistrationDeep Global Registration aligns two 3D scans in three stages: a 6-D convolutional network scores each putative feature match as inlier or outlier, a differentiable Weighted Procrustes gives a closed-form pose from those scores, and a Huber-loss optimiser on SE(3) refines it. On 3DMatch it reports 91.3% recall against 74.9% for RANSAC with 8M iterations (author-reported).Paper · 2020-04-22Universal Dependencies v2: An Evergrowing Multilingual Treebank CollectionDescribes version 2 of the Universal Dependencies guidelines: one cross-lingual scheme for word segmentation, lemmas, universal POS tags, morphological features and dependency relations, plus enhanced graphs. It lists the changes from v1 and the v2.5 release (157 treebanks, 90 languages). A resource paper with no model results.Paper · 2020-04-22Up or Down? Adaptive Rounding for Post-Training QuantizationAdaRound replaces round-to-nearest for post-training weight quantization. Each weight is rounded up or down to minimise a layer-wise reconstruction loss, relaxed with a rectified sigmoid and a rounding regulariser. It uses about 1024 unlabelled images and no fine-tuning. Author-reported: 4-bit weights on Resnet18 reach 68.60% top-1 against 69.68% FP32 and 23.99% for nearest rounding.Paper · 2020-04-21Making Monolingual Sentence Embeddings Multilingual using Knowledge DistillationExtends a monolingual sentence encoder to new languages: a multilingual student (XLM-R) is trained with MSE to map each source sentence and its translation to the teacher's vector for the source. Author-reported: with a paraphrase teacher, STS 2017 averages 84.6 (monolingual) and 83.7 (cross-lingual), above LASER, mUSE and LaBSE; BUCC mining trails LASER and LaBSE (88.6 vs 93.0 and 93.5 F1).Paper · 2020-04-20LSQ+: Improving low-bit quantization through learnable offsets and better initializationLSQ+ extends learned-step-size quantization with a trainable offset next to the learned scale, so skewed activations such as Swish use all 2^b levels without a sign bit. Weight scales come from Gaussian statistics and activation parameters from MSE minimisation, which lowers run-to-run variance. Author-reported: EfficientNet-B0 W4A4 73.8% (LSQ 71.9%, FP 76.1%) on ImageNet.Paper · 2020-04-20StereoSet: Measuring stereotypical bias in pretrained language modelsStereoSet is a crowdsourced English benchmark of 16,995 context association tests in gender, profession, race and religion. Each test offers a stereotype, an anti-stereotype and an unrelated continuation. Models are scored on language modeling and stereotype preference, combined as ICAT. Author-reported best test ICAT: GPT-2 at 73.0 (ideal 100).Paper · 2020-04-20The State and Fate of Linguistic Diversity and Inclusion in the NLP WorldMeasures how NLP venues include languages. Six resource classes come from labelled (LDC, ELRA) and Wikipedia counts. Per-venue language entropy, class-wise inverse MRR and author-language entity embeddings show low-resource classes near the bottom. Author-reported: 549 of 1,139 WALS categories occur only in classes 0-2; LREC and workshops are most inclusive.Paper · 2020-04-19DeepPurpose: A Deep Learning Library for Drug-Target Interaction PredictionOpen-source Python library for drug-target interaction (DTI) prediction. A compound encoder and a protein encoder each produce a vector; the concatenated vectors feed an MLP that predicts a binding score or an interaction probability. Author-reported on DAVIS: CNN+CNN reaches MSE 0.254 and concordance index 0.879, against 0.262 and 0.870 for DeepDTA.Paper · 2020-04-15SPECTER: Document-level Representation Learning using Citation-informed TransformersSPECTER embeds a paper from its title and abstract with SciBERT, further trained with a triplet margin loss so that papers linked by citation sit closer than random or cited-of-cited papers. Embeddings need no fine-tuning or citation data at inference. Author-reported: average 80.0 on the 12 SciDocs metrics, against 76.9 for the next-best baseline.Paper · 2020-04-13Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New OutlooksA survey of knowledge distillation (KD) and student-teacher learning for vision. It explains the temperature-softened KD loss, sorts methods by teacher count, knowledge type, data availability, online and self distillation, learning metric and application, and lists open challenges. It has no experiments and no numeric results of its own.Paper · 2020-04-04BAE: BERT-based Adversarial Examples for Text ClassificationBAE is a black-box attack on text classifiers. It ranks words by importance, masks each one, and uses BERT's masked-LM guesses as replacements or insertions, filtered by sentence similarity (USE) and part of speech. Author-reported: it lowers accuracy more than TextFooler in most settings (BERT with replacement only is the exception) and rates more natural in a small human study.Paper · 2020-04-01Semantic Drift Compensation for Class-Incremental LearningSemantic Drift Compensation (SDC) for class-incremental learning with embedding networks. Prototypes stored after each task are shifted by a Gaussian-weighted average of the embedding drift seen on current-task data. On six-task CUB-200-2011, E-EWC+SDC reaches 56.4% vs 54.2% for E-EWC (author-reported).Paper · 2020-03-29GPS-Net: Graph Property Sensing Network for Scene Graph GenerationOne-stage scene graph generator. Direction-aware message passing uses a tri-linear attention, a loss whose focusing parameter depends on each node's share of triplets, and a frequency prior softened by log-softmax and gated per pair. Author-reported: mean recall 45.9 on VG with the MOTIFS detector, 47.7 with RelDN's.Paper · 2020-03-27Kernel Operations on the GPU, with Autodiff, without Memory OverflowsKeOps evaluates reductions of symbolic kernel or distance formulas, such as Gaussian matrix-vector products, on GPU without storing the full matrix, and supports autodiff. Author-reported: a Gaussian product over 1M 3D points takes 1.39 s on an RTX 2080 Ti, while PyTorch runs out of memory from 100k points.Paper · 2020-03-25Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?RFS: train one ResNet-12 with plain cross-entropy on all meta-training classes merged, freeze it, and fit an L2-normalized logistic regression on each test episode's support features. Born-again self-distillation adds 2-3 points. Reported 64.82/82.14% on miniImageNet 5-way 1/5-shot, beating MetaOptNet (62.64/78.63) with no meta-learning.Paper · 2020-03-10Curriculum Learning for Reinforcement Learning Domains: A Framework and SurveySurvey that formalises curriculum learning for RL as three parts (task generation, sequencing, transfer) and classifies existing methods along seven dimensions. It runs no experiments; its tables categorise published work, and it lists open problems.Paper · 2020-03-06"Other-Play" for Zero-Shot CoordinationOther-play (OP) trains self-play agents under random relabellings of the game's known symmetries, drawn per agent each episode, so they do not depend on arbitrary shared conventions. Added to SAD in Hanabi, cross-play with independently trained agents rises from 17.65 to 22.07 for SAD+AUX; 20 humans scored 15.75 with the OP bot vs 9.15 with the self-play bot. All author-reported.Paper · 2020-03-02Natural Language Processing Advancements By Deep Learning: A SurveySurvey of deep learning in NLP: word and character representations, seq2seq training with its exposure-bias and metric-mismatch problems, and models and benchmarks for tasks from POS tagging to dialogue. It proposes no method. Headline scores, copied from cited works: 97.96 POS accuracy on WSJ-PTB, 95.1 constituency accuracy, 93.09 NER F1 on CoNLL-2003, 35.0 BLEU on English-German.Paper · 2020-02-29First Order Motion Model for Image AnimationAnimates a still object image with a driving video's motion, without object annotations. Learns ten self-supervised keypoints with local affine (2x2 Jacobian) transforms, a dense motion network with an occlusion mask, and a generator trained with perceptual and equivariance losses. Author-reported: lower reconstruction error and user-preferred results than Monkey-Net and X2Face.Paper · 2020-02-15Federated Learning with Matched AveragingFedMA aggregates client CNNs and LSTMs layer by layer: it matches neurons, channels or hidden states across clients with a Bayesian nonparametric Hungarian solver (BBP-MAP), averages matched ones, lets the global layer grow, then clients retrain the next layers with it frozen. VGG-9 on heterogeneous CIFAR-10 (16 clients): 87.53% vs 86.29% FedAvg, with a 1.11x larger model.Paper · 2020-02-11fastai: A Layered API for Deep Learningfastai is a PyTorch-based deep learning library with a layered API: one-call training for vision, text, tabular and collaborative filtering, plus data blocks, two-way callbacks, a generic optimizer, metrics and low-level transforms. Headline results are author-reported: a course rewritten on it and ResNet-50 on ImageNet in 18 minutes (DawnBench); no benchmark tables.Paper · 2019-12-19BERTje: A Dutch BERT ModelBERTje is a Dutch BERT-base model pretrained on 12 GB (about 2.4B tokens) of Dutch books, news, Wikipedia and SoNaR text. It uses sentence order prediction instead of next sentence prediction and masks 15% of tokens. Author-reported: it beats multilingual BERT on every test column of the paper's tables, for NER, POS, semantic roles, spatio-temporal relations and sentiment.Paper · 2019-12-18SynSin: End-to-end View Synthesis from a Single ImageSynSin renders new views of a scene from one RGB image, trained end-to-end on real image pairs with no 3D supervision. It predicts a feature point cloud and a depth map, splats the points with a differentiable renderer, and refines the result with a GAN. Author-reported PSNR: 20.91 on Matterport3D, 22.31 on RealEstate10K, 21.94 on Replica.Paper · 2019-12-14On the Apparent Conflict Between Individual and Group FairnessPhilosophical argument, with no experiments, that individual and group fairness measures do not conflict in principle. Both fit consistency and egalitarian justice, neither meets individual justice, and conflicts arise from unstated worldview assumptions about the source of disparities. Choose the assumption first, then the measure.Paper · 2019-12-11FlauBERT: Unsupervised Language Model Pre-training for FrenchPre-trains a BERT-style masked language model on 71 GB of cleaned French text, in base (138M) and large (373M) sizes, and evaluates it on FLUE, a new six-dataset French benchmark. Author-reported: both sizes outperform multilingual mBERT on most tasks; large has the top score on CLS, dependency parsing and both WSD tasks.Paper · 2019-12-05Trident: Efficient 4PC Framework for Privacy Preserving Machine LearningFour-party secure computation with one active corruption, switching between arithmetic, boolean and garbled sharings over a 64-bit ring for ML training and prediction. A verified 4PC multiplication costs 3 ring elements online, truncation needs no circuit, and ReLU and sigmoid run in constant rounds. Author-reported against malicious ABY3: up to 251.84x faster linear-regression training on LAN.Paper · 2019-12-04StarGAN v2: Diverse Image Synthesis for Multiple DomainsOne generator translates images across several domains. Each domain gets a style code from a mapping network (random noise) or a style encoder (a reference image), injected with AdaIN. Style-reconstruction, diversity and cycle losses keep the outputs diverse and faithful. Author-reported latent-guided FID: 13.7 on CelebA-HQ and 16.2 on AFHQ.Paper · 2019-11-26Image2StyleGAN++: How to Edit the Embedded Images?Embeds a photo into StyleGAN's W+ latent space, then into per-layer noise maps that restore high-frequency detail. Masks make the embedding local, and copying activation tensors combines two embeddings for edits. Author-reported PSNR rises from about 20 dB to 39-45 dB on reconstruction, and masked inpainting beats partial and gated convolution at three resolutions.Paper · 2019-11-25Rigging the Lottery: Making All Tickets WinnersRigL trains a sparse network from a random start with fixed parameter count and fixed training FLOPs. Every 100 steps it drops the smallest-magnitude active weights and grows the same number where the dense gradient is largest, initialised to zero. Author-reported: 80% sparse ResNet-50 reaches 74.6% top-1 at 0.23x training FLOPs, against 76.8% dense.Paper · 2019-11-16Inductive Relation Prediction by Subgraph ReasoningGraIL predicts a missing knowledge-graph triple (u, r, v) from the subgraph between u and v, using no entity embeddings, so it generalizes to unseen entities. Nodes get double-radius distance labels; a relation-aware GNN with edge attention scores the subgraph. Author-reported: best inductive AUC-PR on all 12 splits.Paper · 2019-11-13Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language AnalysisICCN takes outer products of a BERT text vector with LSTM-encoded audio and video states, runs each through a 2D CNN, and trains the two CNNs with a deep CCA loss so their outputs correlate; the outputs join the text vector for a small downstream MLP. Author-reported: best on MOSI in all five metrics (83.07% binary accuracy, MAE 0.862).Paper · 2019-11-10CamemBERT: a Tasty French Language ModelCamemBERT is a monolingual French RoBERTa (BASE, 110M params) pretrained with whole-word masked language modeling on 138GB of web-crawled OSCAR text. It beats mBERT, XLM and XLM-R BASE on French POS tagging, parsing, NER and XNLI (82.5% acc). A 4GB web-crawl sample performs about as well as the full 138GB, and web data beats Wikipedia.Paper · 2019-11-09ConveRT: Efficient and Accurate Conversational Representations from TransformersConveRT is a compact dual encoder for conversational response selection, pretrained on Reddit with an in-batch softmax loss. It combines subword embeddings, a small shared Transformer, 8-bit embeddings and 16-bit network weights. Author-reported: 71.8 R100@1 on Reddit (multi-context), 71.2 R100@1 on DSTC7-Ubuntu (multi-context), 84.3 R100@1 on fine-tuned AmazonQA.Paper · 2019-11-08Certified Data Removal from Machine Learning ModelsCertified removal for L2-regularised linear and logistic models: one Newton step removes a training point's influence, and random loss perturbation at training time hides the leftover error, giving (epsilon, delta)-indistinguishability from a model that never saw the point. Author-reported: removal is 250x to 870x cheaper than retraining on LSUN and SST features.Paper · 2019-11-08ERASER: A Benchmark to Evaluate Rationalized NLP ModelsBenchmark of seven NLP datasets with human-marked rationale snippets, plus metrics for rationales: agreement with humans (IOU F1, token F1, AUPRC) and faithfulness (comprehensiveness, sufficiency, AOPC). Baselines: Lei-style hard selection, BERT-to-BERT pipelines, soft attention scorers. Best IOU F1 is BERT-to-BERT on FEVER, 0.835 (author-reported).Paper · 2019-10-29Federated Learning over Wireless Networks: Convergence Analysis and Resource AllocationEach device minimises a local surrogate that adds a global gradient estimate scaled by eta, solved only to accuracy theta. Linear convergence is proven for smooth, strongly convex losses; the wireless energy-time allocation splits into three sub-problems with closed-form or numerical solutions. Author-reported: lower training loss and higher test accuracy than FedAvg.Paper · 2019-10-28Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationDirichlet calibration maps a classifier's probability vector to calibrated class probabilities. It log-transforms the probabilities, then applies one linear layer and softmax, fitted by log-loss with L2 or off-diagonal and intercept regularisation. On 21 UCI datasets with 11 classifiers, Dirichlet L2 had the best average log-loss rank (author-reported).Paper · 2019-10-27PRNet: Self-Supervised Learning for Partial-to-Partial RegistrationPRNet aligns two partial 3D point clouds with a self-supervised, iterative network: DGCNN and Transformer features, top-k keypoints by feature norm, Gumbel-Softmax keypoint matching with a learned temperature, then an SVD pose. Author-reported on ModelNet40: rotation MSE 10.235 vs 45.005 for DCP-v2.Paper · 2019-10-25A Survey on Recent Advances in Named Entity Recognition from Deep Learning modelsSurvey of named entity recognition from classic feature-based and semi-supervised systems to neural word, character and word+character taggers, comparing 83 selected systems on CoNLL 2002/2003 and SemEval 2013 DrugNER. Adding frequent prefix and suffix embeddings to a word+character model gives author-reported F1 of 87.26 (Spanish), 87.54 (Dutch), 90.86 (English) and 79.01 (German).Paper · 2019-10-23Self-attention for raw optical Satellite Time Series ClassificationCompares a Transformer encoder with an LSTM, two CNNs (MS-ResNet, TempCNN), DuPLO and a random forest on crop-type classification from Sentinel-2 series, raw and preprocessed. Preprocessing helps every model. On raw data the Transformer and LSTM beat the CNNs, and attention scores focus on a few observations rather than clouds.Paper · 2019-10-16Deep Semantic Segmentation of Natural and Medical Images: A ReviewSurvey of deep semantic segmentation for natural and medical images, grouped into architectures, loss functions, synthesis-based data, weakly supervised and multi-task models. It compares losses on toy overlap cases and tabulates datasets and PASCAL VOC 2012 mean IoU. The authors report no experiments of their own; all numbers are taken from other papers.Paper · 2019-10-16The Heidelberg spiking datasets for the systematic evaluation of spiking neural networksTwo speech datasets already converted to spikes: Spiking Heidelberg Digits (recorded digits) and Spiking Speech Commands, made by a cochlea-inspired audio-to-spike model. Spike-count SVMs reach about 60% on SHD; LSTMs reach about 86-89%; a recurrent SNN reaches 71.4%, or 83.2% with tuning. Author-reported.Paper · 2019-10-07DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm SystemDexPilot teleoperates a KUKA arm with an Allegro hand from bare-hand observation by four RGB-D cameras. DART tracking is seeded by neural keypoint and joint priors, and a weighted fingertip-vector cost retargets the hand via SLSQP. Two pilots ran 15 tasks with five trials each; the appendix table shows 9 failed trials out of 150. Author-reported.Paper · 2019-10-02Exploiting BERT for End-to-End Aspect-based Sentiment AnalysisPuts one simple task layer (linear, GRU, self-attention, transformer or CRF) on a fine-tuned BERT-base to tag aspect spans and their sentiment in one pass (E2E-ABSA). Author-reported: BERT with a linear layer already beats the best non-BERT baseline on LAPTOP and REST F1 (60.43 vs 60.35; 73.22 vs 72.78); best variants reach 61.12 and 74.72.Paper · 2019-09-30Domain Adaptation for Semantic Segmentation with Maximum Squares LossUnsupervised domain adaptation for segmentation replaces entropy minimization with a maximum squares loss (negative sum of squared probabilities on unlabelled target pixels), adds image-wise class weights and a low-level head trained on confident pseudo-labels. DeepLabv2 with ResNet-101 reaches 46.4 mIoU on GTA5-to-Cityscapes, against 42.2 for entropy minimization (author-reported).Paper · 2019-09-28Multi-scale Attributed Node EmbeddingAE and MUSAE embed graph nodes with Skip-gram with negative sampling over (node, feature-of-a-random-walk-neighbour) pairs, pooled over distances 1..t (AE) or one model per distance, concatenated (MUSAE). They provably factorize node-feature PMI matrices, contain DeepWalk and Walklets as special cases, and lead unsupervised baselines on most of 8 classification and 3 regression datasets.Paper · 2019-09-26Unsupervised Domain Adaptation through Self-SupervisionTrains a shared network on the source task plus label-free puzzles (rotation, flip, patch location) using source and unlabelled target batches, with no adversarial loss. Early stopping uses a feature-mean distance plus source error. Author-reported: best on 4 of 7 digit and natural-image benchmarks; competitive on GTA5 to Cityscapes.Paper · 2019-09-14Blind Super-Resolution Kernel Estimation using an Internal-GANEstimates the blur kernel of one low-resolution image by training a small linear-generator GAN on that image alone, so the learned downscaler matches the image's own patch statistics. Author-reported: with ZSSR as the SR step, 30.36 dB / 0.867 SSIM at x2 and 26.81 dB / 0.732 SSIM at x4 on 100 DIV2KRK images.Paper · 2019-09-13Neural Oblivious Decision Ensembles for Deep Learning on Tabular DataNODE stacks densely connected layers of differentiable oblivious decision trees. Each tree uses entmax (alpha=1.5) soft feature selection and soft thresholds, and the whole model is trained end-to-end with gradient descent. Author-reported: beats default CatBoost and XGBoost on all six tabular datasets, and tuned CatBoost and XGBoost on four of six.Paper · 2019-09-11Getting Gender Right in Neural Machine TranslationPrepends a speaker-gender tag to the English source of each Europarl sentence and trains LSTM NMT models with and without it for 10 language pairs. Author-reported: significant BLEU gains for French (+1.44), Italian (+0.29) and Danish (+0.31), a significant loss for Finnish (-0.47), and about +1.2 BLEU on EN-FR female-speaker test sets.Paper · 2019-08-30DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in ConversationDialogueGCN classifies the emotion of each utterance in a conversation. A BiGRU encodes the utterances in order, then a two-layer relational graph convolution over a windowed utterance graph adds speaker context, with edge types set by speaker pair and temporal order. Authors report 64.18 weighted F1 on IEMOCAP (DialogueRNN: 62.75) and 58.10 F1 on MELD.Paper · 2019-08-28Optimal transport mapping via input convex neural networksLearns the optimal transport map for quadratic cost from samples: two input convex neural networks f and g are trained in a minimax game, and the transport map is the gradient of g. Author-reported: on 784-dimensional Gaussian pairs the transported mean lands close to the target mean for shifts alpha = 1, 5, 10 (Table 1).Paper · 2019-08-27Sentence-BERT: Sentence Embeddings using Siamese BERT-NetworksSentence-BERT fine-tunes BERT or RoBERTa in a siamese setup: both sentences pass through the same network, are mean-pooled into one vector, and are compared by cosine similarity. Trained on NLI, STS or triplets. Author-reported: SBERT-NLI-large averages 76.55 Spearman over seven STS tasks; the closest pair among 10,000 sentences takes about 5 s instead of about 65 h with cross-encoder BERT.Paper · 2019-08-26Once-for-All: Train One Network and Specialize it for Efficient DeploymentOne super-network whose sub-networks vary in depth, width, kernel and input size. Progressive shrinking trains the largest first, then fine-tunes smaller choices. Per-device sub-networks are picked by an accuracy predictor and latency table, with no retraining. Author-reported: 80.0% ImageNet top-1 under 600M MACs after fine-tuning; 76.0% at 230M MACs without it, vs 75.2% MobileNetV3-Large.Paper · 2019-08-22EGNet: Edge Guidance Network for Salient Object DetectionSalient object detection with a U-Net-style object path and a separate salient-edge path, trained jointly with an edge loss. The edge features are added to each object-scale feature before prediction, and the four guided maps are fused. Author-reported, no post-processing: max F-measure 0.893 on DUTS-TE with ResNet and 0.880 with VGG.Paper · 2019-08-13Attention is not not ExplanationTests whether attention weights in LSTM text classifiers can count as explanations, with a uniform-attention baseline, a seed-variance baseline, a frozen-weight MLP guide and an adversary that diverges from base attention. Adversarial attention exists on some binary tasks but guides an MLP poorly (author-reported: F1 0.503 vs 0.753 on Diabetes).Paper · 2019-08-02AutoML: A Survey of the State-of-the-ArtSurvey of AutoML organised along the ML pipeline: data preparation, feature engineering, hyperparameter optimization and, mainly, neural architecture search (search spaces; EA, RL, gradient, surrogate, random and hybrid optimizers; cheap evaluation). Tabulates published NAS results on CIFAR-10 and ImageNet with GPU-day costs and discusses one-shot, one-stage and resource-aware NAS.Paper · 2019-07-27Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentBlack-box word-substitution attack for text classifiers and NLI models. It ranks words by how much their removal shifts the prediction, then swaps in embedding synonyms that pass a sentence-similarity check and a part-of-speech check. Author-reported: BERT's SNLI accuracy falls from 89.4% to 4.0% with 18.5% of words changed.Paper · 2019-07-27Multi-task Self-Supervised Learning for Human Activity DetectionSelf-supervised pre-training for accelerometer activity recognition: a 1-D CNN with eight heads learns which signal transform (noise, scaling, sign flip, reversal, permutation, time warp, channel shuffle) was applied. Its conv features then train an activity classifier. Kappa is 0.76-0.89 on six datasets, above autoencoders and near supervised. No official code found.Paper · 2019-07-24Benchmarking TPU, GPU, and CPU Platforms for Deep LearningIntroduces ParaDnn, a parameterized suite of end-to-end FC, CNN and RNN training models, and benchmarks Cloud TPU v2/v3, a V100 GPU and a Skylake CPU with six real models. Each platform wins somewhere: TPU for large batches and CNNs, GPU for small batches and large FCs, CPU for the largest models. Author-reported: TPU 3-6.8x faster than GPU on real models.Paper · 2019-07-22BEHRT: Transformer for Electronic Health RecordsBEHRT is a BERT-style Transformer over UK primary-care diagnosis histories (CPRD, about 1.6M patients, 301 diagnosis codes). Each visit is a sentence of codes with age, segment and position embeddings. It is pre-trained by masked disease prediction, then fine-tuned for multi-label next-visit and 6/12-month prediction. Author-reported: 8.0-10.8 points higher average precision than DeepR and RETAIN.Paper · 2019-07-22FedHealth: A Federated Transfer Learning Framework for Wearable HealthcareFedHealth builds a cloud CNN from federated updates sent under homomorphic encryption, so the authors say raw wearable data is not shared. Each user freezes the convolution layers and fine-tunes the fully connected layers with a CORAL loss. On five held-out UCI Smartphone subjects, the authors report 99.4% mean accuracy against 94.1% for the cloud model alone.Paper · 2019-07-19Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample DatasetsAn LSTM variant, the Entity-Aware-LSTM (EA-LSTM), feeds static catchment attributes only into its input gate, so one network learns rainfall-runoff for 531 CAMELS basins. Trained with a basin-normalised NSE loss, one model reaches a mean validation NSE of 0.67 and an 8-model ensemble 0.70, above the benchmark models compared (author-reported).Paper · 2019-07-15Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale BenchmarksD3Net predicts salient objects from RGB and depth with three VGG-16 streams (RGB, RGB-D, depth). At test time a depth depurator keeps the RGB-D map only if it agrees with the depth-only map within 0.15 MAE, else uses RGB. Adds the SIP phone-captured dataset and an 18-model benchmark. Author-reported: S-measure 0.900 on NJU2K test (CPFP 0.879).Paper · 2019-07-11Making AI Forget You: Data Deletion in Machine LearningTwo k-means variants that remove one training point without retraining from scratch. Q-k-means snaps centroids to a random-phase lattice, so most deletions leave them unchanged. DC-k-means clusters random leaf subsets and recomputes one leaf and the root. Author-reported: amortized runtime 2.2x to 584x below k-means++ retraining across six datasets; loss ratio up to 1.44.Paper · 2019-07-01Time-to-Event Prediction with Neural Networks and Cox RegressionReplaces the Cox model's linear risk term with a neural network, trained with SGD on a case-control approximation of the partial likelihood (one sampled control per event). Cox-Time adds time as an input, dropping proportional hazards. Author-reported on KKBox: Cox-Time has the best integrated Brier score and log-likelihood; DeepHit has the best concordance.Paper · 2019-06-30Variational Quantum Circuits for Deep Reinforcement LearningReplaces the neural Q-network of deep Q-learning with a variational quantum circuit: 4 qubits, 2 layers, the state encoded in bits, actions read from Pauli-Z expectations plus a trainable bias. The 28-parameter circuit reaches 0.9 reward on frozen lake and 100 on cognitive-radio channel selection in simulation; a short IBM hardware run matches the simulator (author-reported).Paper · 2019-06-19XLNet: Generalized Autoregressive Pretraining for Language UnderstandingXLNet pretrains a Transformer-XL with permutation language modeling: each sequence gets a random factorization order, and a two-stream attention lets the model predict a token from its position without seeing its content. This gives bidirectional context without [MASK] tokens. With BERT's data and recipe it beats BERT on all 11 tasks the authors compared (author-reported).Paper · 2019-06-14Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous DrivingTrains a stereo network on depth instead of disparity (depth cost volume, depth loss), then corrects its depth map with a few 4-beam LiDAR points by graph-based propagation. On KITTI val (car, PointRCNN, IoU 0.7, moderate) it raises AP_BEV from 56.0 to 76.9 with the sparse LiDAR; author-reported, with the 4-beam LiDAR simulated from 64-beam data.Paper · 2019-06-13The Replica Dataset: A Digital Replica of Indoor SpacesReplica is a dataset of 18 indoor 3D scene reconstructions with dense meshes, 16-bit HDR textures, per-primitive semantic labels (88 classes) and mirror and glass reflectors, built with a custom RGB-D rig, SLAM and manual annotation. The paper reports no learning benchmark, only a comparison with Matterport3D and ScanNet. The data is non-commercial.Paper · 2019-06-11Weight Agnostic Neural NetworksEvolves network topologies without training any weight: every connection shares one value, and each topology is scored over several such values, ranked by reward and connection count. Author-reported: with one tuned shared weight, swing-up reaches 723 against 8 for a fixed network, car racing 608 against -37, and MNIST about 92% test accuracy.Paper · 2019-06-04Exact Combinatorial Optimization with Graph Convolutional Neural NetworksLearns branch-and-bound variable selection: a bipartite graph convolutional network imitates strong branching, trained by cross-entropy over candidate variables. On four NP-hard MILP families in SCIP 6.0.1 it is most accurate and, in 11 of 12 size settings, faster than SCIP's default reliability pseudocost rule, including on larger instances (author-reported).Paper · 2019-06-03Evaluating Gender Bias in Machine TranslationWinoMT, a 3,888-sentence English challenge set built by joining Winogender and WinoBias, plus an automatic metric for gender bias in MT. The English entity is aligned to its translation, and its target-side gender is read with per-language morphology rules. Author-reported: all four commercial systems and both academic models show gender bias, and commercial accuracy ranges from 28.9% to 74.1%.Paper · 2019-05-29Are Disentangled Representations Helpful for Abstract Visual Reasoning?Tests whether disentangled representations help abstract visual reasoning. Builds two Raven-style puzzle sets from dSprites and 3dshapes, trains 360 unsupervised disentanglement encoders, then 3600 Wild Relation Networks on their embeddings. Author-reported: FactorVAE scores track reasoning accuracy best with few training samples; the link weakens by 100K steps.Paper · 2019-05-29Disentangling Monocular 3D Object DetectionMonocular 3D car detection from one calibrated RGB image with a ResNet34-FPN two-stage detector. Each 3D box loss is split into copies that each free one parameter group, and a self-supervised 3D confidence ranks boxes. Also fixes a flaw in KITTI's 11-point AP. Author-reported: 3D AP40 moderate 7.94 on KITTI test (larger training split), up from 5.74 for MonoGRNet.Paper · 2019-05-15Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and ReasoningSurvey of neural-symbolic computing, which pairs neural learning with symbolic knowledge, reasoning and explanation. Organises methods by representation (rules, formulas, tensors/embeddings, temporal logic), learning (ILP, hybrid learning), reasoning (chaining, approximate SAT) and explainability (rule extraction, program synthesis). No experiments of its own, so no new numbers.Paper · 2019-05-15Transferable Clean-Label Poisoning Attacks on Deep Neural NetsConvex Polytope (CP) attack: clean-label poisons whose features, in several substitute networks, enclose the target's features, so a victim that overfits them labels the target as the poison class. Substitutes use Dropout while crafting; end-to-end victims need a multi-layer variant. Author-reported: over 50% transfer success at 1% poisoning; feature collision stays at or below 50%.Paper · 2019-05-13BayesNAS: A Bayesian Approach for Neural Architecture SearchBayesNAS treats one-shot architecture search as Bayesian sparse learning. Each architecture weight gets a hierarchical ARD prior tied to its predecessor edges, the weights are updated after one epoch of training, and edges whose prior variance is at most 1/(2πe) are pruned. Reports 2.81% CIFAR-10 test error at about 0.2 GPU days (author-reported, with cutout).Paper · 2019-05-10Survey on Evaluation Methods for Dialogue SystemsSurvey of how dialogue systems are evaluated, by class: task-oriented, conversational and interactive QA. Covers task success and cost metrics, the PARADISE and Interaction Quality satisfaction models, user simulation, learned metrics (ADEM, RUBER) and Recall@k, plus datasets. No new experiments; its PARADISE pooled test R² is 0.50.Paper · 2019-05-08Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent AdvancesSurvey of emotion recognition in conversation (ERC): why context, speaker identity, emotion shift, sarcasm and emotion reasoning make it hard; six public datasets with label and split statistics; and recent models (CMN, ICON, IANN, DialogueRNN, HRLCE). Most reported results come from earlier papers. Headline: DialogueRNN, 62.75 weighted F1 on IEMOCAP. No new method.Paper · 2019-05-08Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial ImagesAdapts a segmenter trained on Potsdam aerial images to Vaihingen without target labels. A CycleGAN-style unpaired translator restyles the labelled source images to the target sensor look, then the segmenter is fine-tuned on them. Author-reported overall accuracy on Potsdam to Vaihingen rises from 0.35 to 0.52; the gain is concentrated in the Building and Tree classes.Paper · 2019-05-02Full-Gradient Representation for Neural Network VisualizationFull-gradients split a ReLU net's output exactly into input-gradient and bias-gradient terms. FullGrad, for CNNs, aggregates post-processed versions of these into one saliency map. Author-reported: strongest among compared methods on pixel perturbation and remove-and-retrain.Paper · 2019-04-30MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood MixingMixHop replaces the graph convolution layer with one that concatenates features propagated over several adjacency powers (default A^0, A^1, A^2), each with its own weights, so a layer can learn differences between neighbourhoods at different distances. On Planetoid splits, the learned variant scores 71.4, 81.9 and 80.8 percent against 70.3, 81.5 and 79.0 for vanilla GCN.Paper · 2019-04-29Style Transfer by Relaxed Optimal Transport and Self-SimilaritySTROTSS restyles a content image by optimisation. Style is a relaxed earth mover's distance over VGG16 features, plus moment and palette terms. Content is the self-similarity of features, so layout survives while pixels change. Users can pin region pairs. A 662-worker study finds higher style scores than prior methods for a given content score (author-reported; no numeric table).Paper · 2019-04-28RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape CompletionA DDPG agent picks the latent seed of a latent GAN that turns a noisy, partial point-cloud encoding into a clean one, which an autoencoder decodes into a completed shape. Author-reported: about 1 ms per completion instead of a 324 s optimisation batch, and PointNet accuracy on 70% incomplete ShapeNet inputs rising from 50.2% to 83.8% (hybrid).Paper · 2019-04-17Text Classification Algorithms: A SurveySurvey of text classification as a pipeline: cleaning and features (bag-of-words, TF-IDF, Word2Vec, GloVe, FastText), dimensionality reduction (PCA, LDA, NMF, autoencoders, t-SNE), classifiers (Rocchio, boosting, SVM, trees, CRF, deep models) and metrics (F-beta, MCC, ROC, AUC). No experiments; qualitative comparisons only.Paper · 2019-04-16Detecting the Unexpected via Image ResynthesisFlags unexpected objects in street scenes without training on them. A segmenter labels the image, a pix2pixHD generator repaints it from that label map, and a discrepancy network marks pixels where the two images disagree, trained on random label swaps of known objects. Author-reported AUROC 0.79-0.93 on Lost and Found and a 60-image road set, above uncertainty and autoencoder baselines.Paper · 2019-04-16Real Image Denoising with Feature AttentionRIDNet is a single-stage blind denoiser for real photographs. It stacks four enhancement attention modules with residual-on-residual skip connections and channel feature attention, trained with an L1 loss. Author-reported: 39.23 dB PSNR and 0.9526 SSIM on the DnD sRGB benchmark, blind, versus 38.06 dB for CBDNet.Paper · 2019-04-13Texture image analysis and texture classification methods - A reviewReview of texture image analysis and classification. Sorts descriptors into statistical, structural, model-based and transform-based families and describes GLCM, LBP, Gabor, fractal and combined methods, the classifiers used and benchmark datasets. Runs no experiments of its own; its accuracy table collects author-reported numbers on different datasets and protocols.Paper · 2019-04-10Fitness Dependent Optimizer: Inspired by the Bee Swarming Reproductive ProcessFDO is a PSO-like swarm optimizer whose step size comes from a fitness-derived weight instead of a velocity term, with a random factor and a reused pace after successful moves. Author-reported: it beats PSO, GA and DA on 19 classical functions and DA, WOA and SSA on 10 CEC-2019 functions, with 30 runs, 10-D.Paper · 2019-04-06Doodle to Search: Practical Zero-Shot Sketch-based Image RetrievalZero-shot sketch-based image retrieval: a sketch retrieves photos of unseen classes. Two VGG-16 encoders with attention, trained with a triplet loss, a gradient-reversal domain loss and a semantic loss against word2vec class vectors. Adds QuickDraw-Extended (330k sketches, 204k photos). Author-reported full-gallery mAP: 0.3691 Sketchy, 0.1094 TU-Berlin, 0.0752 QuickDraw.Paper · 2019-04-05An Attentive Survey of Attention ModelsSurvey of attention in neural networks. Frames attention as A(q,K,V) = sum_i p(a(k_i,q)) v_i with a pluggable alignment function a and distribution function p, tabulates 12 alignment functions, sorts attention types along four axes (sequences, abstraction levels, positions, representations) and reviews architectures, applications and interpretability. No experiments.Paper · 2019-04-02Semantics Disentangling for Text-to-Image GenerationSD-GAN, a stacked text-to-image GAN. A Siamese discriminator pulls together images generated from two captions of the same picture and pushes apart images from different pictures (contrastive loss). Semantic-conditioned batch norm modulates generator features from sentence and word cues. Author-reported Inception score: CUB 4.67 vs 4.36 (AttnGAN), MS-COCO 35.69 vs 25.89.Paper · 2019-03-29Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning SystemsA concept, a three-axis taxonomy (knowledge source, representation, pipeline integration point) and a survey of informed machine learning: training with prior knowledge such as equations, logic rules, simulations, invariances, knowledge graphs, probabilistic relations and human feedback. Survey only: no experiments and no quantitative results of its own.Paper · 2019-03-28FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic SegmentationFastFCN replaces the dilated convolutions of a segmentation backbone with Joint Pyramid Upsampling (JPU): a learned module that upsamples the last three ResNet stages to output stride 8 using parallel dilated separable convolutions. Author-reported: 51.05 mIoU on PASCAL-Context (ResNet-50, single scale) at 37.56 FPS, against 49.91 mIoU and 18.77 FPS for dilated EncNet.Paper · 2019-03-26All about Structure: Adapting Structural Information across Domains for Boosting Semantic SegmentationDISE splits each image into a domain-invariant structure code, used for segmentation, and a domain-specific texture code, trained with reconstruction, texture-swap translation and adversarial losses. Swapped images carry source labels to the target domain. Author-reported 45.4 mIoU on GTA5 to Cityscapes, against 42.4 for the output-space adaptation baseline.Paper · 2019-03-25Recurrent Back-Projection Network for Video Super-ResolutionRBPN super-resolves a video frame by feeding each neighbouring frame, with its optical flow to the target, through a recurrent encoder-decoder. Each step mixes a single-image upscaler with a multi-frame path and refines the HR features, which are then concatenated for reconstruction. At 4x on SPMCS-11 it reports 30.10 dB (RBPN/6-PF), above VSR-DUF's 29.42 dB. Author-reported.Paper · 2019-03-20LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous DrivingLaserNet detects 3D objects from LiDAR in the sensor's native range view with one fully convolutional network. Each point predicts a mixture of box distributions, merged by mean-shift clustering and filtered by an adaptive NMS that uses the predicted variance. Author-reported: 85.34 vehicle AP0.7 (0-70 m) on ATG4D and 30 ms total runtime on KITTI; it trails BEV detectors on KITTI.Paper · 2019-03-10Group-wise Correlation Stereo NetworkBuilds the stereo matching cost volume by splitting feature channels into groups and correlating each group at every disparity, then refines it with an improved stacked 3D hourglass network. Author-reported: outperforms the compared methods on Scene Flow, KITTI 2012 and KITTI 2015; KITTI 2015 test D1-all 2.11% vs 2.32% for PSMNet.Paper · 2019-03-09Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove ThemArgues that two word-embedding debiasing methods (Hard-Debiased, GN-GloVe) hide gender bias rather than remove it. After debiasing, the most biased words still cluster by gender, neighbour counts still track the original bias, and an RBF SVM still predicts gender. Author-reported SVM accuracy: 98.25% to 88.88% (Hard-Debiased), 98.65% to 96.53% (GN-GloVe).Paper · 2019-03-01A Review of Stochastic Block Models and Extensions for Graph ClusteringReview of stochastic block models (SBMs) for graph clustering. Classifies variants by graph type (binary, valued, signed), degree correction, soft versus hard membership, inference (MCMC, variational, greedy) and how the number of groups is chosen. Covers dynamic and topic-model extensions and compares models in four tables. Runs no experiments of its own.Paper · 2019-02-27On Tiny Episodic Memories in Continual LearningExperience Replay (ER) trains each step on the current mini-batch plus a same-size batch from a tiny episodic memory (about one example per class), in a single pass over each task stream. At that memory size it beats fine-tuning, EWC, A-GEM and MER on four split benchmarks. Author-reported CIFAR accuracy: 56.2% with ring-buffer writing vs 54.9% for A-GEM.Paper · 2019-02-20ScispaCy: Fast and Robust Models for Biomedical Natural Language ProcessingA set of spaCy-based biomedical pipelines: tokenizer rules, joint POS tagging and dependency parsing, NER and sentence segmentation, plus a character-trigram nearest-neighbour candidate generator for UMLS concepts. Authors report NER competitive with strong baselines on 5 of 9 datasets, and parsing 2-3 points below a Biaffine parser at about nine times its speed. Author-reported.Paper · 2019-02-15GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face ReconstructionFits a 3D face model to one photo by gradient descent. A GAN trained on UV face textures supplies the skin texture prior, and a pretrained face-recognition network supplies the identity and content losses. Author-reported mean point-to-plane error on MICC (cooperative setting) is 0.95 for the method against 1.50 for the best baseline, and it recovers high-frequency texture.Paper · 2019-02-04Attention in Natural Language ProcessingSurvey of attention in NLP. It defines one general model: a compatibility function scores keys against a query, a distribution function (usually softmax) turns scores into weights, and the weighted values form a context vector. A four-axis taxonomy covers input representation, compatibility, distribution and multiplicity. It has no experiments.Paper · 2019-01-23Sentiment and Sarcasm Classification with Multitask LearningTrains one GRU network with attention and a neural tensor network to classify sentiment and sarcasm jointly, so each task helps the other. On a 994-sentence benchmark with 10-fold CV, the best variant reaches average F-score 86.66 against 82.13 for the prior state of the art (author-reported).Paper · 2019-01-21CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert ComparisonCheXpert is a dataset of 224,316 chest radiographs labeled for 14 observations as positive, negative or uncertain, with labels extracted from reports by a rule-based tool. A DenseNet121 study compares uncertain-label handling methods. On a 500-study test set, the model's ROC and PR curves lie above all 3 radiologists' operating points on Cardiomegaly, Edema and Pleural Effusion (author-reported).Paper · 2018-12-29Greedy Layerwise Learning Can Scale to ImageNetTrains a CNN one block at a time. Each block (3x3 conv, ReLU, optional invertible downsampling) is fit with a shallow auxiliary classifier on the frozen output of earlier blocks, with no end-to-end backprop. One-hidden-layer blocks reach 79.7% top-5 single-crop on ImageNet (above AlexNet); 3-hidden-layer blocks reach 88.7% (89.8% ensemble). Author-reported.Paper · 2018-12-03Deep Learning for Classical Japanese LiteratureThree character-image datasets cut from 18th-century cursive Japanese books: Kuzushiji-MNIST (10 classes), Kuzushiji-49 (49) and Kuzushiji-Kanji (3,832). Baselines: 4-NN, a small CNN, PreActResNet-18 with mixup. Domain transfer: two VAEs and a mixture density network map Kuzushiji-Kanji latents to Modern Kanji strokes, qualitative only. Headline: 98.83% on Kuzushiji-MNIST (author-reported).Paper · 2018-12-03Towards Accurate Generative Models of Video: A New Metric & ChallengesProposes Fréchet Video Distance (FVD): fits Gaussians to I3D (Kinetics-400) features of real and generated video clips and takes the FID-style Fréchet distance. In a human study on BAIR, FVD agrees with raters better than SSIM and PSNR. Also introduces StarCraft 2 Videos (SCV), four benchmark scenarios, with baseline FVD scores for four video models.Paper · 2018-12-02Efficient Lifelong Learning with A-GEMA-GEM replaces GEM's per-task gradient constraints with one constraint on the average episodic-memory gradient: if the current gradient g has negative dot product with a reference gradient g_ref from a random memory batch, project g onto g_ref's orthogonal plane. Matches GEM accuracy (62.3% vs 61.2% on Split CIFAR) at roughly 10x less training time.Paper · 2018-12-02ProxylessNAS: Direct Neural Architecture Search on Target Task and HardwareSearches CNN architectures directly on ImageNet or CIFAR-10 and the target hardware, with no proxy task. Binarized gates keep one sampled path per edge in memory; a differentiable expected-latency term puts hardware cost into the loss. Author-reported: 2.08% CIFAR-10 error with 5.7M parameters; 74.6% ImageNet top-1 at 78 ms on a Pixel 1, about 200 GPU hours.Paper · 2018-11-29Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPoseAdapts bottom-up OpenPose to run in real time on CPUs. A dilated MobileNet v1 backbone, a lightweight refinement block and faster post-processing cut the network to 9 GFLOPs, about 15% of the 2-stage baseline, with 42.8 AP against 43.4 for the baseline (author-reported). Reported 28 fps on an Intel NUC and 26 fps on an i7-6850K CPU.Paper · 2018-11-19Scalable agent alignment via reward modeling: a research directionPosition paper proposing reward modeling for agent alignment: learn a reward function from user feedback, then train an RL agent on it, both at once. It lists five scaling challenges (feedback amount, feedback distribution, reward hacking, unacceptable outcomes, reward-result gap), pairs them with approaches, and describes recursive reward modeling. It reports no experiments.Paper · 2018-11-15Reward learning from human preferences and demonstrations in AtariLearns a reward model from pairwise preferences over 25-step Atari clips, optionally with automatic labels from expert demonstrations, then trains a DQfD agent on that learned reward. Beats imitation learning in 7 of 9 games (abstract) and reaches superhuman scores on Pong and Enduro. Author-reported; synthetic labels from the game score, plus some human labels.Paper · 2018-11-14YOLO-LITE: A Real-Time Object Detection Algorithm Optimized for Non-GPU ComputersYOLO-LITE is a 7-layer, batch-norm-free variant of Tiny-YOLOv2 for real-time object detection on laptops and phones without a GPU. Found by trial and error on PASCAL VOC, it reaches about 34% mAP on VOC (author-reported, figures vary across the paper) at 21 FPS on a non-GPU laptop, and 12.26% mAP on COCO.Paper · 2018-11-10Breast Cancer Classification from Histopathological Images with Inception Recurrent Residual Convolutional Neural NetworkIRRCNN is a CNN whose blocks combine inception-style multi-size branches, recurrent convolutional layers unrolled over time steps, and a residual add, followed by transition blocks and softmax. Trained with SGD on BreakHis and the 2015 BCC challenge data, it reports 97.95% image-level binary accuracy on BreakHis at 40x and 99.05% image-wise binary accuracy on BCC 2015. Author-reported.Paper · 2018-11-01DialogueRNN: An Attentive RNN for Emotion Detection in ConversationsTracks a GRU state per speaker through a conversation, plus a global GRU for context and an emotion GRU for the utterance representation, with attention over earlier context. A bidirectional variant with attention over all emotion states is the best model. Author-reported: 63.40% weighted accuracy on IEMOCAP vs 56.56% for CMN, and gains on all four AVEC attributes.Paper · 2018-10-19Synscapes: A Photorealistic Synthetic Dataset for Street Scene ParsingSynscapes: 25,000 path-traced synthetic street-scene images with Cityscapes-style labels, instance and depth maps, and per-actor metadata. Cityscapes-trained DeepLab v3+ reaches 63.64 mean IoU on it, against 45.84 on a GTA-based dataset. Pre-training on Synscapes and fine-tuning on Cityscapes gives 78.85 vs 76.56 (author-reported).Paper · 2018-10-09DeepWeeds: A Multiclass Weed Species Image Dataset for Deep LearningReleases DeepWeeds, 17,509 in-field photos of eight Australian rangeland weed species plus a Negatives class, with five stratified train/val/test folds. ImageNet-pretrained ResNet-50 and Inception-v3 fine-tuned with augmentation reach 95.7% and 95.1% average accuracy. ResNet-50 on TensorRT (Jetson TX2) takes 53.4 ms per image. Author-reported.Paper · 2018-10-09Warped Hypertime Representations for Long-term Autonomy of Mobile RobotsAdds time to spatial robot models by projecting time onto circles whose periods are found with FreMEn. The resulting spatio-temporal space is clustered with Gaussian mixtures to predict p(a | x, t). Experiments on four robot datasets report better predictions than Mean, histogram and FreMEn baselines. No numeric results appear in the text, so the ranking comes from figures only.Paper · 2018-10-08Deep learning cardiac motion analysis for human survival predictionTurns right-ventricle motion from cardiac MRI (202 tracked points over 20 frames, 11,514 values) into a latent code with a denoising autoencoder trained with a Cox partial-likelihood survival loss. Author-reported internal bootstrap: optimism-corrected C-index 0.73 vs 0.59 for a Cox model on manual RV volumes, in 302 patients from one centre.Paper · 2018-10-08Meta-Learning: A SurveySurvey of meta-learning, organised by the meta-data used: past model evaluations, task meta-features, and prior models (transfer, learned optimisers, MAML-style initialisation, few-shot learners). Gives a shared formulation, the method families and their trade-offs, and the scale of cited studies. Reports no new experiments.Paper · 2018-10-05HG-DAgger: Interactive Imitation Learning with Human ExpertsHG-DAgger is a DAgger variant for human experts. The human alone decides when to take control, and labels are collected only during those takeovers. It also learns a doubt threshold from the takeovers, using ensemble variance. On simulated and real-vehicle driving, the author-reported results beat DAgger and behavioral cloning on road departures and collisions.Paper · 2018-10-05MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in ConversationsMELD is a multimodal, multi-party emotion dataset: about 13,000 utterances from the TV series Friends, with aligned audio, video and text, labelled with 7 emotions and 3 sentiments. Baselines (text-CNN, bcLSTM, DialogueRNN) use dialogue context and audio. Best reported weighted F-score: 60.25 for 7-way emotion (DialogueRNN, text+audio) and 67.56 for sentiment. Author-reported.Paper · 2018-10-02CINIC-10 is not ImageNet or CIFAR-10CINIC-10 is a 270,000-image dataset of 32x32 pictures in CIFAR-10's ten classes, built by adding ImageNet images to CIFAR-10. It has three equal 90,000-image splits and works as a drop-in CIFAR-10 replacement. Author-reported test error for six CIFAR-style CNNs ranges from 8.55 (ResNeXt29_2x64d) to 18.00 (MobileNet).Paper · 2018-09-09A Neural Temporal Model for Human Motion PredictionPredicts long human motion from motion capture with a two-level GRU: a backward-running plan RNN, driven by a noise sample and the action label, feeds guide vectors to a forward body RNN. A multi-objective loss ramps from open- to closed-loop prediction to curb drift. Proposes NPSS, a power-spectrum metric checked by a user study. Author-reported: lowest NPSS on three of four Human3.6M actions.Paper · 2018-09-04Recurrent World Models Facilitate Policy EvolutionTrains a VAE to encode 64x64 frames, an LSTM with a 5-mixture density head (MDN-RNN) to predict the next latent, and a linear controller on latent plus hidden state, evolved with CMA-ES. Reports 906 ± 21 on CarRacing-v0 and a controller trained only inside the learned model scoring 1092 ± 556 on DoomTakeCover-v0 (author-reported).Paper · 2018-08-29Decoupling Strategy and Generation in Negotiation DialoguesSplits a negotiation bot into a manager that picks coarse dialogue acts such as propose(price=150) and a retrieval generator that writes the sentence. Act-level RL keeps human-likeness at 2.8 to 3.0 of 5 on CraigslistBargain, where word-level RL falls to 1.7 to 1.9 (author-reported).Paper · 2018-08-22Don't Use Large Mini-Batches, Use Local SGDPost-local SGD runs ordinary mini-batch SGD until the first learning-rate decay, then switches to local SGD: each worker takes H local steps between model averages. Author-reported: on CIFAR-10 with 16 workers and global batch 2048, ResNet-20 reaches 93.02% top-1 test accuracy (H=32), against 92.48% for large-batch SGD and 92.63% for small-batch SGD.Paper · 2018-08-21CoQA: A Conversational Question Answering ChallengeCoQA is a dataset of 127k question-answer turns from 8k conversations about passages in seven domains. Answers are free-form, each with a highlighted text-span rationale. Baselines include a span reader (DrQA), a copy-based generator and a combination; the best reaches 65.4 test F1 against 88.8 for humans (author-reported).Paper · 2018-07-31Deep learning in agriculture: A surveySurvey of 40 studies, mostly from 2015 to 2017, that apply deep learning to agricultural and food production tasks such as weed, fruit and plant recognition, land cover and crop type classification. It tabulates architectures, data sources, preprocessing, metrics and comparisons. Most classification studies report accuracy above 90%, but the studies are rarely directly comparable.Paper · 2018-07-19Visual Domain Adaptation with Manifold Embedded Distribution AlignmentMEDA maps source and target features onto a Grassmann manifold with a geodesic flow kernel, then fits a closed-form kernel classifier that aligns marginal and per-class MMD. An adaptive weight mu, estimated from A-distances, balances the two terms. Author-reported: 73.2% mean accuracy over 28 tasks (Office+Caltech10, USPS/MNIST, ImageNet/VOC) vs 69.7% for JGSA.Paper · 2018-07-18On Evaluation of Embodied Navigation AgentsConsensus recommendations from a working group on evaluating navigation agents in 3D: report goal type and sensors, quantify prior exploration, add an explicit DONE action, measure geodesic distance, use SPL as the primary score, and benchmark on standard SUNCG, Matterport3D, AI2-THOR and Gibson scenarios. The paper runs no new experiments.Paper · 2018-07-09Weakly-Supervised Convolutional Neural Networks for Multimodal Image RegistrationA 3D CNN predicts a dense displacement field that aligns T2 MR to 3D transrectal ultrasound of the prostate. It is trained only on sparse anatomical label pairs, using a multiscale soft Dice label similarity with bending-energy regularisation. Inference needs just the image pair and runs in real time. Cross-validation on 108 pairs: median TRE 3.6 mm and median gland Dice 0.87 (author-reported).Paper · 2018-06-21Quantizing deep convolutional networks for efficient inference: A whitepaperPractical guide to 8-bit and 4-bit integer quantization of CNNs with uniform affine/symmetric quantizers. Post-training per-channel weight plus per-layer activation quantization stays within about 2% of float on ImageNet. Quantization-aware training (STE, batch-norm correction and freezing) closes the gap to about 1% at 8 bits. Quantized inference is 2-3x faster on a Pixel 2 CPU.Paper · 2018-06-20Automated Fact Checking: Task formulations, methods and future directionsSurvey of automated fact checking from NLP and related fields. It unifies task formulations by input (claim sentences, knowledge-base triples, documents), output (labels, scores, labels with evidence) and evidence type (none, metadata, knowledge graphs, text, social-media signals). It reviews datasets, methods and open challenges. It reports no experiments of its own.Paper · 2018-06-16Multimodal Sentiment Analysis using Hierarchical Fusion with Context ModelingFuses text, audio and video utterance features in two stages: pairs of modalities are compared per dimension and contextualised with a GRU, then the three pairs are fused and passed through another GRU. Author-reported: 1-2 points over early fusion and prior CMU-MOSI and IEMOCAP methods.Paper · 2018-06-13SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health ConditionsSMHD builds a Reddit dataset of users who self-report one of nine mental health diagnoses, found with high-precision diagnosis patterns and synonym expansion, plus matched control users chosen on post count and subreddit overlap. Author-reported: 20,406 diagnosed and 335,952 control users. Best baseline FastText reaches F1 41 to 58 per condition on binary tasks and is best on 7 of 9.Paper · 2018-06-10All-in-one: Multi-task Learning for Rumour VerificationTrains one LSTM with stance and rumour-detection heads alongside a veracity head, sharing the recurrent layers, on Twitter reply branches (PHEME, RumourEval). On PHEME with 5 events the three-task model reaches macro-F 0.396 against 0.339 for the NileTMRG* baseline; on RumourEval the veracity-plus-stance model reaches 0.558 against 0.539 for NileTMRG*. Author-reported.Paper · 2018-06-01Implicit Bias of Gradient Descent on Linear Convolutional NetworksGradient descent on a full-width linear convolutional network of depth L, with exponential loss on separable data, converges in direction to a stationary point of the Fourier-domain l_{2/L} bridge penalty, so depth changes the bias. Fully connected linear networks converge to the hard-margin SVM for any depth. Theory only; no experiments.Paper · 2018-05-16PACT: Parameterized Clipping Activation for Quantized Neural NetworksPACT replaces ReLU with a clipped activation whose clip level alpha is a learned, per-layer parameter trained by backpropagation with an L2 penalty. Activations are then uniformly quantized to k bits with a straight-through estimator. Author-reported: 4-bit weights and activations stay within about 1 point of full precision top-1 on CIFAR10, SVHN, AlexNet, ResNet18 and ResNet50.Paper · 2018-05-13Learning Rich Features for Image Manipulation DetectionRGB-N: a two-stream Faster R-CNN that boxes tampered regions. An RGB stream (ResNet-101) proposes RoIs and regresses boxes; a noise stream sees the image through three fixed SRM high-pass filters. RoI features of both streams are fused by compact bilinear pooling for classification. Pre-trained on synthetic COCO splices; F1 0.722 and AUC 0.937 on NIST16 (author-reported).Paper · 2018-04-19Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research ChallengesSurvey of civil UAV applications (search and rescue, remote sensing, infrastructure inspection, agriculture, delivery, traffic monitoring, surveillance, wireless coverage) with market figures, air-to-ground path-loss models and open challenges in charging, collision avoidance, networking and security. Not a method paper: no experiments. Market numbers are cited industry forecasts.Paper · 2018-04-18Deep Transfer Network with Joint Distribution Adaptation: A New Intelligent Fault Diagnosis Framework for Industry ApplicationDeep transfer network (DTN) trains a CNN on labelled source fault data, then adapts it to unlabelled target data with a joint distribution adaptation term: an MMD over all features plus one per class on pseudo-labels. On ten transfer tasks across three datasets, DTN with JDA averages 97.5% vs 80.0% for a plain CNN (author-reported).Paper · 2018-04-12Regularisation of Neural Networks by Enforcing Lipschitz ContinuityCaps the Lipschitz constant of each layer (its worst-case stretch under l1, l2 or l-inf norm) and keeps it under the cap by projecting weights after every update. The network bound is the product of layer caps. Author-reported: CIFAR-10 VGG19 rises from 90.43% to 92.57% with LCC-l2, and gains are largest with small training sets.Paper · 2018-04-11Attention U-Net: Learning Where to Look for the PancreasAdds additive attention gates to the skip connections of a 3D U-Net. Each gate scores voxels from the skip features and a coarser gating signal, then rescales the features with sigmoid coefficients, so no separate localisation model is needed. Author-reported on CT pancreas segmentation: pancreas Dice 0.840 vs 0.814 for U-Net on the 120/30 CT-150 split.Paper · 2018-04-09Speech Commands: A Dataset for Limited-Vocabulary Speech RecognitionSpeech Commands v2: 105,829 one-second 16 kHz clips of 35 English words from 2,618 speakers, crowd-recorded on phone and laptop mics, filtered by size, loudness and human review, with hash-based stable splits. Defines a 12-class Top-One protocol (10 words + unknown + silence) and a streaming metric; the TensorFlow conv baseline reaches 88.2% Top-One.Paper · 2018-04-05Inferring transportation modes from GPS trajectories using a convolutional neural networkClassifies travel mode (walk, bike, bus, driving, train) from raw GPS trajectories with a 1-D CNN over 200-point segments. Each point gets four channels: speed, acceleration, jerk and bearing rate. A seven-model ensemble reaches 84.8% test accuracy on GeoLife, author-reported; the test set also guided epoch selection.Paper · 2018-04-01Training Tips for the Transformer ModelTraining recipe for the Transformer in Tensor2Tensor, tested on English-to-Czech with 58M sentence pairs. Covers batch size, GPU count, model size, max sentence length, learning rate with warmup and checkpoint averaging, judged on full BLEU learning curves. With 8 GPUs, a fixed learning_rate effectively rises by sqrt(8). The best 8-GPU model reaches BLEU 23.8 on WMT17 en-cs.Paper · 2018-03-27World ModelsA VAE compresses each frame into a latent z, an MDN-RNN predicts the next z from actions and a hidden state h, and a single-layer controller on [z, h] is evolved with CMA-ES. Car Racing: 906 ± 21 over 100 trials, above the 838 leaderboard entry (author-reported). The controller is also trained inside the learned dream on VizDoom Take Cover and transferred to the real game.Paper · 2018-03-21Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression NetworkRegresses a 256x256 UV position map, which stores the x, y, z position of every face point, from one face image with a light residual encoder-decoder. No 3DMM is fitted. Trained on 300W-LP with a region-weighted loss. Author-reported: NME 3.62 on AFLW2000-3D (68 points), 9.8 ms per image on a GTX 1080, and over 25% relative gains on alignment and reconstruction.Paper · 2018-03-20Explanation Methods in Deep Learning: Users, Values, Concerns and ChallengesBook-chapter survey of explanation needs for deep neural networks. It names expert and lay users, summarises GDPR provisions on meaningful information, and proposes a three-class taxonomy of explanation methods (rule extraction, attribution, intrinsic) judged by fidelity, interpretability, generalizability and explanatory power. It reports no experiments or numbers.Paper · 2018-03-06Annotation Artifacts in Natural Language Inference DataShows that the hypothesis alone reveals the label in SNLI and MultiNLI. A fastText classifier that never sees the premise scores 67.0% on SNLI and 53.9% and 52.3% on MultiNLI matched and mismatched, against 34-35% for the majority class. Strong NLI models score much lower on the hard subset that this classifier gets wrong.Paper · 2018-03-03Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep LearningU-Net-style networks (U-Net, TernausNet-11/16, LinkNet-34) label each pixel of surgical video frames as robot instrument or background, or as one of 3 parts or 7 instrument types. Trained with cross-entropy plus a soft-Jaccard term. Author-reported best: binary IoU 83.60% (TernausNet-16), parts IoU 65.50% (TernausNet-16), instrument-type IoU 34.61% (TernausNet-11).Paper · 2018-03-03The History Began from AlexNet: A Comprehensive Survey on Deep Learning ApproachesSurvey of deep learning as of 2018: DNNs, CNNs (LeNet to ResNet and DenseNet), RNNs with LSTM and GRU, autoencoders, RBMs, GANs, deep RL, transfer learning, efficient hardware and frameworks. It collects the core equations and training algorithms. Headline: ImageNet top-5 error of 3.57% for ResNet-152, as cited. No new method and no code.Paper · 2018-03-02Essentially No Barriers in Neural Network Energy LandscapeConnects independently trained minima of CNNs, ResNets and DenseNets on CIFAR-10/100 using AutoNEB, a nudged elastic band with adaptive pivot insertion. Author-reported: loss stays near the minima's loss along the paths. For the deepest CIFAR-10 models the saddle training loss is close to the minimum (DenseNet-100-12-BC: 0.0026 vs 0.0005). Deep ResNets lose at most 0.5 test-error points.Paper · 2018-02-28Model-Based Value Estimation for Efficient Model-Free Reinforcement LearningModel-based value expansion (MVE) rolls a learned one-step model forward a fixed H steps from replayed states and adds the critic's value at the end. Its TD-k trick trains the critic on those imagined states, each with horizon H minus t. Results are learning curves only: the author-reported gain over DDPG and an imagination-buffer baseline holds on all three tasks, and is smaller on walker.Paper · 2018-02-26A Robust Real-Time Automatic License Plate Recognition Based on the YOLO DetectorFour-stage ALPR pipeline built from YOLO-style detectors: vehicle detection, plate detection, character segmentation, then separate letter and digit recognizers, with negative-image and flipped-character augmentation and per-vehicle majority voting. Author-reported: 93.53% on SSIG at 47 FPS and 78.33% on the new UFPR-ALPR dataset at 35 FPS, ahead of OpenALPR and Sighthound.Paper · 2018-02-22xView: Objects in Context in Overhead ImageryxView is an overhead object-detection dataset from WorldView-3 imagery at 0.3 m GSD: over 1 million boxes in 60 classes over 1,400+ km², built with three-stage quality control. An SSD baseline trained on multi-resolution chips (300/400/500 px) reaches total mAP 0.2590, against 0.1456 for vanilla 300 px chips (author-reported).Paper · 2018-02-21Continual Lifelong Learning with Neural Networks: A ReviewReview of catastrophic forgetting in neural networks and of biological lifelong learning. Groups fixes into regularization (LwF, EWC, SI), dynamic architectures (progressive networks, growing self-organizing maps) and replay with complementary memories. No new experiments; headline comparisons are qualitative and taken from cited work.Paper · 2018-02-06Multi-Temporal Land Cover Classification with Sequential Recurrent EncodersBidirectional convolutional GRU (or LSTM) encoder over Sentinel-2 time series: 30 random observations per training step, the final cell states of both passes feed a per-pixel softmax classifier, and no cloud masking is applied. Author-reported: 89.7% overall accuracy and kappa 0.870 on 17 crop classes for 2016 (89.5% for 2017).Paper · 2018-02-06Texygen: A Benchmarking Platform for Text Generation ModelsTexygen is an open benchmarking platform for text generation. It bundles seven baseline models (MLE, SeqGAN, MaliGAN, RankGAN, TextGAN, GSGAN, LeakGAN) and five metrics (BLEU, EmbSim, NLL-oracle, NLL-test, Self-BLEU) on a synthetic LSTM oracle and COCO captions. Author-reported: LeakGAN has the highest test BLEU on COCO, and MLE the lowest Self-BLEU.Paper · 2018-02-02Deep Convolutional Neural Networks for Breast Cancer Histology Image AnalysisClassifies H&E breast histology images with frozen ImageNet CNN features (ResNet-50, InceptionV3, VGG-16) pooled over 50 colour and 20 crop augmentations per image, then averages LightGBM models over 10 folds and 5 seeds. Author-reported: 87.2% four-class and 93.8% carcinoma-vs-non-carcinoma accuracy (ICIAR 2018 data).Paper · 2018-01-31In Defense of Classical Image Processing: Fast Depth Completion on the CPUFills the holes in sparse LIDAR depth maps with a fixed chain of OpenCV steps (inversion, shaped dilation, hole closing, hole fill, median and Gaussian blur). No training data and CPU-only at about 90 Hz. Author-reported KITTI depth completion test RMSE 1288.46 mm, ranked first among published methods at submission.Paper · 2018-01-29Road Damage Detection Using Deep Neural Networks with Images Captured Through a SmartphoneRoad Damage Dataset 2018: 9,053 dashboard-smartphone images (600×600) from 7 Japanese municipalities with 15,435 boxes in 8 damage classes (cracks, potholes, faded markings), PASCAL VOC format. SSD MobileNet and SSD Inception V2 at 300×300 serve as baselines; per-class recall ranges from 0.02 to 0.89; MobileNet runs in 30.6 ms on a GPU and 1.5 s on a Nexus 5X.Paper · 2018-01-23ArcFace: Additive Angular Margin Loss for Deep Face RecognitionArcFace L2-normalizes face embeddings and class weights, adds an angular margin m=0.5 to the target angle and scales logits by s=64 before softmax. Sub-center ArcFace (K=3 centers per class) isolates noisy web faces so they can be dropped automatically. Trained on the cleaned IBUG-500K set, ResNet100 reaches 97.27% TPR@FPR=1e-4 on IJB-C (author-reported).Paper · 2018-01-17TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image SegmentationAdds an ImageNet-pretrained VGG11 encoder to a U-Net for binary building segmentation on aerial images, trained with binary cross-entropy minus log soft-Jaccard. Author-reported validation IoU after 100 epochs: 0.593 with LeCun-uniform initialisation, 0.686 with the pretrained encoder, 0.687 with a fully pretrained network (Carvana). Official MIT PyTorch code.Paper · 2017-12-18DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density EstimationDecideNet counts people by blending a detection density map (Faster R-CNN head boxes as Gaussian bumps) with a regression density map (5-layer CNN). A 4-layer attention net (QualityNet) weights the two per pixel and is trained with a detection-score quality term. Author-reported: lowest MAE on Mall (1.52) and WorldExpo'10 (9.23 average); MAE 21.53 on ShanghaiTech PartB.Paper · 2017-12-13Predicting Station-level Hourly Demands in a Large-scale Bike-sharing Network: A Graph Convolutional Neural Network ApproachGCNN-DDGF predicts next-hour demand at each of 272 Citi Bike stations without a predefined station graph: the graph filter is learned from data inside a graph CNN. A recurrent (LSTM) variant scores best: author-reported test RMSE 2.12 vs 2.35 for the regular variant and 3.44 for the historical-average baseline.Paper · 2017-12-06Joint 3D Proposal Generation and Object Detection from View AggregationAVOD fuses a LiDAR bird's-eye-view map and an RGB image in full-resolution feature pyramids. A region proposal network proposes 3D boxes, and a second stage refines them and predicts orientation. Author-reported: the highest KITTI test car 3D AP among the compared methods at all three difficulty levels (66.38% at hard), at about 0.1 s per frame.Paper · 2017-12-04FSSD: Feature Fusion Single Shot Multibox DetectorFSSD adds a feature fusion module to SSD: 1x1-reduced conv4_3, fc7 and conv7_2 maps are resized to 38x38, concatenated, batch-normalised and used to build a new feature pyramid for the detection heads. FSSD300 reaches 78.8 mAP on VOC2007 test at 65.8 FPS on a 1080Ti (VOC07+12 training); 82.7 with COCO pre-training (author-reported).Paper · 2017-11-30Towards Accurate Binary Convolutional Neural NetworkABC-Net trains binary CNNs. Each layer's real weights are approximated by a linear combination of M binary weight bases, and its activations by N binary activations with trainable shifts, so a convolution becomes M×N bitwise convolutions. Author-reported ImageNet Top-1 with ResNet-18: 65.0% at M=N=5 (full precision 69.3%), against 51.2% for XNOR-Net.Paper · 2017-11-29Road Extraction by Deep Residual U-NetDeep ResUnet: a 7-level U-Net-style encoder-decoder built from full pre-activation residual units, with no cropping, trained with MSE and SGD on 224x224 patches. On the Massachusetts roads test set it reports a relaxed break-even point of 0.9187 against U-Net's 0.9053, with about a quarter of U-Net's parameters (author-reported).Paper · 2017-11-22On the Automatic Generation of Medical Imaging ReportsGenerates chest x-ray reports with a multi-task model: a VGG-19 tag classifier predicts medical keywords, a co-attention module attends jointly over image regions and the embeddings of the top-10 tags, and a hierarchical LSTM emits one topic per sentence, then the words. Author-reported BLEU-4 0.247 and CIDEr 0.327 on IU X-Ray, above re-implemented captioning baselines.Paper · 2017-11-12AON: Towards Arbitrarily-Oriented Text RecognitionRecognises words in natural images whatever their orientation or curvature. Four 1D feature sequences (left-right, right-left, top-bottom, bottom-top) are weighted per position by learned placement clues, fused, and decoded by an attention LSTM, trained end-to-end on word labels. Author-reported: best lexicon-free accuracy on irregular CUTE80 (76.8) and ICDAR15 (68.2), comparable on regular sets.Paper · 2017-10-23VGGFace2: A dataset for recognising faces across pose and ageVGGFace2 is a face recognition training set of 3.31M web images of 9,131 identities (80 to 843 per identity), built by a six-stage automatic and manual filtering pipeline that targets pose and age variety and low label noise. ResNet-50 and SE-ResNet-50 trained on it, optionally after MS-Celeb-1M pre-training, report 0.921 TAR@FAR=1e-3 on IJB-A (author-reported).Paper · 2017-10-19Be Your Own Prada: Fashion Synthesis with Structural CoherenceGenerates a new outfit on a person's photo from a text description, keeping pose and body shape. Stage 1 generates a clothing segmentation map from a coarse 8x8 merged map of the wearer; stage 2 renders each region's texture via a per-label compositional mapping. Author-reported attribute mAP 82.6 vs 71.8-74.7 for baselines.Paper · 2017-09-24Long Text Generation via Adversarial Training with Leaked InformationLeakGAN trains a text GAN whose generator has a Manager LSTM that reads the discriminator's CNN features at every step and emits a goal vector. A Worker LSTM uses the goal to choose each next word. Author-reported: lower synthetic NLL than SeqGAN and RankGAN at lengths 20 and 40, and BLEU-2 0.956 vs 0.859 for SeqGAN on EMNLP2017 WMT News.Paper · 2017-09-15Search-based Motion Planning for Quadrotors using Linear Quadratic Minimum Time ControlPlans collision-free, dynamically feasible, minimum-time quadrotor paths by A* over a lattice of short constant-input motion primitives. Each primitive is a closed-form LQMT solution, and the LQMT cost is a tighter heuristic than a minimum-time bound. Author-reported on one start state: 376 expanded nodes with the LQMT heuristic vs 2707 for Dijkstra.Paper · 2017-09-07Focusing Attention: Towards Accurate Text Recognition in Natural ImagesAttention-based scene text recognizer plus a focusing network that pulls each character's attention back onto the character. Part of the training data has character pixel labels. Uses a 32-layer ResNet-style encoder. Author-reported lexicon-free accuracy: 87.4% IIIT5k, 85.9% SVT, 94.2% IC03, 93.3% IC13, 70.6% IC15, above the same model without the focusing network.Paper · 2017-09-04Machine learning methods for histopathological image analysisMini-review of machine learning for digital histopathology (whole-slide images). It walks through the patch-based pipeline, then lists recurring problems (very large images, few labels, magnification, stain colour variation, artifacts) with the methods proposed for each. No experiments; the outlook covers outlier detection, interpretability, intraoperative use and immune-cell analysis.Paper · 2017-09-03A Generative Model For Zero Shot Learning Using Conditional Variational AutoencodersZero-shot image classification with a conditional VAE trained on image features given class attribute or word2vec vectors. It samples pseudo-features for unseen classes and fits an SVM on them. Reports the best per-class accuracy on AwA1, AwA2 and SUN (disjoint), a large gain on the generalized AwA sets, and 24.7 top-5 on ImageNet. Author-reported.Paper · 2017-09-03Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic LossRefineGAN reconstructs MRI images from under-sampled k-space with a residual encoder-decoder GAN. A cyclic data-consistency loss ties the output back to the measurements, and a second chained generator refines it. Reported: about 0.1 s per 256x256 image and over 42 dB average PSNR at 40% sampling (author-reported).Paper · 2017-08-30Deep Learning to Improve Breast Cancer Early Detection on Screening MammographyTurns an ROI-trained patch classifier (ResNet50 or VGG16) into an all-convolutional whole-image mammogram classifier with two conv top blocks, global average pooling and no heatmap, trained end-to-end on image labels. Author-reported per-image AUC on DDSM test: 0.91 for a 4-model ensemble; after transfer to INbreast: 0.98.Paper · 2017-08-11Neural Expectation MaximizationNeural EM treats grouping pixels into objects as inference in a spatial mixture model with neural-network components, and unrolls EM into a differentiable, end-to-end trainable clustering. RNN-EM segments synthetic shape sequences and supports next-step prediction. Author-reported: AMI 0.826 on static shapes (N-EM 0.475); 0.970 on three flying shapes.Paper · 2017-08-09Recent Trends in Deep Learning Based Natural Language ProcessingSurvey of deep learning for NLP: word vectors (word2vec, GloVe, FastText, ELMo, BERT), CNNs, RNNs (LSTM, GRU), recursive networks, attention and the Transformer, reinforcement and generative models, memory networks. Collects benchmark tables for tagging, parsing, NER, SRL, sentiment, translation, QA and dialogue. Numbers are the cited papers', not re-measured; no experiments.Paper · 2017-08-02Structure-measure: A New Way to Evaluate Foreground MapsS-measure scores a non-binary foreground/saliency map against a binary ground truth by averaging a region-aware term (SSIM on four blocks split at the GT centroid) and an object-aware term (mean and spread of predicted values inside foreground and background). Authors report it beats AP, AUC and Fbw on most meta-measures across five datasets.Paper · 2017-07-31Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence TaggingRandom seeds alone can move a sequence tagger's test F1 by about one point, so single-run scores mislead. The authors propose comparing score distributions over many runs. Across about 50,000 BiLSTM networks on five tagging tasks, CRF output, Nadam, variational dropout and two stacked layers give the most robust results.Paper · 2017-06-29Grasp Pose Detection in Point CloudsGrasp Pose Detection (GPD): samples 6-DOF two-finger grasp candidates from a depth point cloud, encodes each as a 15-channel 60x60 multi-view image and scores it with a 4-layer CNN. Author-reported: 93% grasp success over 288 attempts in dense clutter on a Baxter robot, 84% with sensors fixed to the body. No CAD model needed.Paper · 2017-06-20Advanced Steel Microstructural Classification by Deep Learning MethodsPixel-wise segmentation with a fully convolutional network (FCN-style, VGG16 backbone, fine-tuned) followed by a per-object majority vote classifies constituents of low-carbon steel in SEM images. Author-reported 93.94% object accuracy on 21 images, against 48.89% for the prior object-based method of Pauly et al.Paper · 2017-06-10Recovery Guarantees for One-hidden-layer Neural NetworksRecovers the weights of a one-hidden-layer regression net from noiseless Gaussian data. Activations with three distilled properties give a positive-definite Hessian near the truth. Tensor initialisation plus resampled gradient descent then recovers the parameters from about d·log(1/ε) samples, up to polynomial factors. Theory, with synthetic squared-ReLU checks.Paper · 2017-06-04Brain Intelligence: Go Beyond Artificial IntelligencePosition paper proposing Brain Intelligence (BI): many small neural sub-networks linked by artificial-life connections that share parameters and structure, with each sub-network updated by an S-system (reproduction, selection, mutation), plus memory and an imagine function. It gives no equations, algorithms or experiments, so its claim that BI would solve four AI limitations is untested.Paper · 2017-05-24Exploring the Regularity of Sparse Structure in Convolutional Neural NetworksPrunes CNN weights at four granularities (single weights, sub-kernel vectors, kernels, filters) by L1 magnitude and compares them on ImageNet. Coarser grains lose top-5 accuracy at equal density, yet store fewer indices and need about 30-35% fewer simulated VGG-16 output memory references than fine-grained pruning. Results are author-reported.Paper · 2017-05-22Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and SegmentationAdds a training-only shape regulariser to CNN cardiac MR/US segmentation and MR super-resolution. An autoencoder (or a predictor for intensity images) maps outputs to 64-d codes, and the loss penalises code distance to ground truth. No inference-time cost, per the authors. Author-reported: endocardial Dice 0.939 vs 0.923 for the sub-pixel baseline; SR SSIM 0.796 vs 0.783 for SR-CNN.Paper · 2017-05-15Key-Value Retrieval Networks for Task-Oriented DialogueAn LSTM encoder-decoder for in-car task-oriented dialogue that adds a key-value attention over knowledge-base triples to the decoder's output logits, so the model emits KB entities without dialogue-state tracking. Trained on a new 3,031-dialogue corpus, it reaches 48.0 entity F1 and 13.2 BLEU on the test set (author-reported), above a rule-based and a copy-augmented baseline.Paper · 2017-05-12Quality and Diversity Optimization: A Unifying Modular FrameworkShows MAP-Elites and Novelty Search with Local Competition are two configurations of one quality-diversity loop with a swappable container (grid or archive) and selection operator. Adds an exclusive epsilon-dominance archive rule and a curiosity score for parent selection. Authors report curiosity selection gives the best collections in three simulated robot tasks.Paper · 2017-05-07Deep Visual Attention PredictionPredicts human eye fixations with a VGG16 encoder and three learned deconvolution decoders on conv3-3, conv4-3 and conv5-3, each with its own cross-entropy loss, merged by a learned fusion layer. Trained on 10,000 SALICON images. Author-reported: highest scores on most metrics for four of five eye-tracking benchmarks (not MIT300), about 0.1 s per image on a GPU.Paper · 2017-04-22Medical Text Classification using Convolutional Neural NetworksClassifies single sentences of medical text into 26 categories with a CNN over word2vec vectors: two pairs of 256-filter conv layers with max pooling, then a 128-unit dense layer and softmax. Reports 0.68 accuracy against 0.28 to 0.51 for doc2vec, mean-embedding and bag-of-words baselines. Author-reported, on a dataset that is not released.Paper · 2017-04-09Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural NetworksTwo-stream RNN for skeleton action recognition. A temporal LSTM stream (stacked or hierarchical, over per-frame joints) and a spatial LSTM stream (joints ordered as a chain or traversal) are fused by weighting class posteriors. Rotation, scaling and shear augmentation. Author-reported: 71.3% and 79.5% on NTU RGB+D cross-subject and cross-view, 94.8% on SBU, 91.7 F1 on ChaLearn.Paper · 2017-04-08A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image ReconstructionReconstructs undersampled cardiac MR images with a cascade of CNN blocks. Each block is followed by a data-consistency layer that keeps the measured k-space samples, and a data-sharing layer borrows k-space from neighbouring frames for dynamic sequences. Author-reported: 2D 3-fold MSE 0.89e-3 vs 2.12e-3 for DLMRI; a 3D sequence in 8.21 s on a GTX 1080.Paper · 2017-04-08Deep Reinforcement Learning framework for Autonomous DrivingProposes an end-to-end deep RL pipeline for driving: sensor fusion, an attention glimpse network feeding a CNN, LSTM temporal integration, and a DQN or DDAC planner. Shows lane keeping in the TORCS simulator, but only qualitatively (no metrics). Attention and recurrence are described, not evaluated.Paper · 2017-04-07Threat analysis of IoT networks Using Artificial Neural Network Intrusion Detection SystemTrains a small three-layer sigmoid MLP (6 inputs, 3 hidden units, 1 output) with backpropagation and a squared-error loss to classify normal versus DDoS/DoS traffic on a simulated five-sensor IoT network. The input features are not named. Author-reported 99.4% accuracy on 3305 samples; the test split has 496 samples.Paper · 2017-03-07Unsupervised Learning of Sentence Embeddings using Compositional n-Gram FeaturesSent2Vec: a sentence vector is the mean of learned unigram and bigram embeddings, trained with a C-BOW-style negative-sampling objective that predicts each word from its sentence. Author-reported: 82.0 average on six supervised tasks (unigram+bigram, Toronto books), second only to ordered-sentence SkipThought, and the best unsupervised similarity average among unordered models.Paper · 2017-02-27Age Progression/Regression by Conditional Adversarial AutoencoderConditional adversarial autoencoder (CAAE) for face age progression and regression. An encoder maps a face to a 50-dim personality vector, a generator renders it at a chosen age bin, and two discriminators (on the vector and on the image) keep codes smooth and faces realistic. No paired ages needed. Evidence is a user study: 52.77% of votes prefer CAAE over prior work, 18.24% judge them equal.Paper · 2017-02-02Video Salient Object Detection via Fully Convolutional NetworksVideo saliency model with a static and a dynamic fully convolutional network. The dynamic net reads two frames plus the static map, so no optical flow is computed. Training uses synthetic videos made by warping annotated still images, plus about 3K real frame pairs. Author-reported MAE: 6.36% on DAVIS, 7.65% on FBMS, about 2 fps.Paper · 2017-01-27Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee CrisisTwo online surveys (56 people, 20 German tweets each) ask whether a hate speech definition makes annotators agree more. It does not: Krippendorff's alpha stays between .18 and .29, while the definition group more often says a tweet should be banned. Author-reported; the paper's analysis code is not released.Paper · 2017-01-23Adversarial Learning for Neural Dialogue GenerationTrains a Seq2Seq dialogue generator with REINFORCE, using a hierarchical-LSTM discriminator's probability that a response is human-written as the reward, plus per-token rewards via Monte Carlo rollouts (REGS) and interleaved MLE "teacher forcing" for stability. Also proposes adversarial evaluation (AdverSuc) with reliability checks. Beats an MMI baseline 0.62 vs 0.18 in human pairwise judgments.Paper · 2017-01-17Joint Deep Modeling of Users and Items Using Reviews for RecommendationDeepCoNN predicts ratings from review text: one CNN reads all reviews a user wrote, a second CNN reads all reviews of the item, and a factorization machine on the concatenated outputs predicts the rating. Author-reported MSE beats MF, PMF, LDA, CTR, HFT and CDL on Yelp, Amazon (21 categories) and Beer, by 7.6% to 8.7% over the best baseline.Paper · 2017-01-10Real-Time Bidding by Reinforcement Learning in Display AdvertisingModels impression-level bidding for one ad campaign as an MDP over auctions left, budget left and request features. Exact value iteration on a small grid gives a bid rule that weighs predicted click value against the value cost of the price. A small tanh network approximates value differences at scale. Author-reported: 16.7% more clicks than a linear bidder at a 1/8 budget (iPinYou).Paper · 2016-12-26Clustering Algorithms: A Comparative ApproachBenchmark, not a new method: compares seven R clustering algorithms (k-means, clara, agnes, EM, hcmodel, spectral, hddc subspace) on 270 synthetic datasets with ARI, Jaccard, Fowlkes-Mallows and NMI. Spectral clustering is best at default settings (ARI 65.80% averaged). Tuning one parameter or random search over all parameters improves several methods.Paper · 2016-12-18Learning a No-Reference Quality Metric for Single-Image Super-ResolutionLearns a no-reference metric that scores single-image super-resolution outputs without ground truth. Trained on 81,000 human ratings of 1,620 SR images, it uses 138 low-level DCT, wavelet and patch-singular-value features fed to three random forests whose outputs are linearly combined. Author-reported Spearman correlation with human scores is 0.931 under 5-fold cross-validation.Paper · 2016-12-16FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomicsFusionNet, a U-Net-style encoder-decoder for segmenting neurons in electron microscopy. Residual blocks sit inside every level and pixel-wise summation skip connections join encoder and decoder levels. Trained on 8x orientation-enriched, elastically deformed tiles. Author-reported: V_rand 0.978 on Drosophila (U-net 0.973); Dice 0.946 on zebrafish (U-net 0.908).Paper · 2016-12-16Medical Image Synthesis with Context-Aware Generative Adversarial NetworksPredicts a CT volume from an MR volume with a patch-based 3D fully convolutional generator trained with an adversarial loss, an L2 loss and a 3D image gradient difference loss, then refines it with two Auto-Context stages that feed the previous CT estimate back as a second input channel. Author-reported PSNR 27.6 (brain) and 34.1 (pelvis), above atlas, sparse-coding and random-forest baselines.Paper · 2016-12-06Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text TranslationTranslates French speech directly into English text with one attention-based encoder-decoder, without a source transcript. Trained on about 16 h of synthetic French speech from six voices (BTEC), it reaches 48.8 BLEU on a held-out speaker's test1 set with a 5-model ensemble, against 52.7 for a Google ASR plus SMT pipeline (author-reported).Paper · 2016-12-01Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D SupervisionPredicts a 32x32x32 voxel grid from one 64x64 image. A perspective projection layer that takes the max over depth turns the grid into silhouettes, compared with silhouettes from 24 views; the method is designed to need no 3D ground truth. Author-reported chair test IoU 0.5027 without volume loss vs 0.5067 with it, and 0.4983 for a volume-only CNN.Paper · 2016-11-29C-RNN-GAN: Continuous recurrent neural networks with adversarial trainingTrains a GAN whose generator and bidirectional-LSTM discriminator model continuous MIDI events (length, frequency, intensity, time since the previous tone), with feature matching and six epochs of pretraining. Author-reported: more varied classical music than a maximum-likelihood baseline; three tones per cell raise polyphony. Results are plots only.Paper · 2016-11-19Invertible Conditional GANs for image editingInvertible cGAN (IcGAN): a conditional GAN plus two encoders that map a real image to a noise vector z and an attribute vector y, so changing y regenerates the image with edited attributes. E_z is trained on generated images, E_y on real ones. Author-reported on CelebA: the independent-encoder setup has the lowest reconstruction loss (0.429 vs 0.500 for shared layers).Paper · 2016-11-07Unsupervised Cross-Domain Image GenerationDomain Transfer Network (DTN): learns G = g∘f mapping unpaired source images to a target domain so a fixed feature extractor f gives the same output before and after. Loss: ternary GAN, f-constancy, target-identity and TV terms. SVHN→MNIST transfer is classified correctly 90.66% of the time; nearest-neighbour domain adaptation reaches 84.44% vs 73.85% for DANN (author-reported).Paper · 2016-11-01Product-based Neural Networks for User Response PredictionProduct-based Neural Network (PNN) predicts clicks from multi-field categorical data: learned field embeddings, a product layer of pairwise inner or outer products, then ReLU layers and a sigmoid. Author-reported AUC: IPNN 77.79% on Criteo and OPNN 81.74% on iPinYou, above every baseline in the paper's tables. The official TensorFlow demo differs from the paper.Paper · 2016-10-27A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural NetworksPre-trained CNNs for sentiment, emotion and personality act as feature extractors. Their features, merged with a baseline CNN's features and fed to an SVM, reach author-reported F1 of 97.71%, 94.80% and 93.30% on three Twitter sarcasm datasets (5-fold CV).Paper · 2016-10-27Ex Machina: Personal Attacks Seen at ScaleCombines crowdsourced labels with a bag-of-n-grams classifier to label 63M English Wikipedia talk comments at scale. The classifier matches the aggregate of 3 crowd-workers on AUC and Spearman, and its machine labels are used to analyse attack prevalence, anonymity, activity and moderation.Paper · 2016-10-19POI: Multiple Object Tracking with High Performance Detection and Appearance FeaturePOI, an online and an offline multi-object tracker built on a Faster R-CNN detector and a GoogLeNet-like 128-d appearance embedding (softmax plus triplet loss). The online tracker uses a Kalman filter and Kuhn-Munkres matching; the offline tracker extends H2T. Author-reported: MOT16 challenge MOTA 66.1 (online) and 68.2 (offline).Paper · 2016-10-13Why Deep Neural Networks for Function Approximation?Proves that ReLU and binary-step networks approximate smooth univariate functions to error eps with polylog(1/eps) neurons when depth grows as log(1/eps), while shallow networks need poly(1/eps) neurons. Also gives matching lower bounds for strongly convex functions and multivariate polynomial and composition bounds. Purely theoretical: no experiments.Paper · 2016-10-06Places: An Image Database for Deep Scene UnderstandingPlaces, a scene-centric image database of about 10.6 million photos labelled with 434 scene categories. Built from web image search, crowdsourced labelling and classifier-assisted bootstrapping. Defines Places205 and Places365 benchmarks and trains AlexNet, GoogLeNet and VGG scene CNNs. Author-reported: Places365-VGG 55.19% top-1 on the Places365 test set.Paper · 2016-09-18SeqGAN: Sequence Generative Adversarial Nets with Policy GradientTrains a sequence generator (an LSTM) as a reinforcement-learning policy over discrete tokens. A CNN discriminator scores each finished sequence, and Monte Carlo roll-outs spread that score back to every token so the policy can be updated by policy gradient. On synthetic data the oracle NLL is 8.736 against 9.038 for MLE (lower is better, author-reported).Paper · 2016-09-14Convolutional Recurrent Neural Networks for Music ClassificationConvolutional recurrent neural network (CRNN) for music tagging on the Million Song Dataset (top-50 tags). Two-dimensional convolutions extract local time-frequency features, and a 2-layer GRU summarises them over time. Compared with three CNNs at equal parameter counts (0.1M to 3M), CRNN has the highest AUC-ROC in every setting; a fully-convolutional CNN (k2c2) is faster to train at equal size.Paper · 2016-08-04Solving General Arithmetic Word ProblemsSolves multi-step arithmetic word problems without templates by predicting a read-once expression tree. Per-quantity relevance and per-pair LCA-operation SVMs feed a beam search under positive-answer and integer-answer constraints. Author-reported accuracy: 78.0% on AI2 (positive constraint only), 73.9% on IL, 45.2% on a new multi-step set.Paper · 2016-07-15Attention-over-Attention Neural Networks for Reading ComprehensionAttention-over-Attention (AoA) Reader for cloze-style reading comprehension. It weights query-to-document attention by a learned document-to-query attention instead of fixed merge rules. An optional N-best re-ranking step adds n-gram LM features. Author-reported: 74.4% CNN test, 72.0% CBTest NE test and 69.4% CBTest CN test for one model.Paper · 2016-06-15Progressive Neural NetworksProgressive networks add a new column per task, freeze earlier columns and feed their features in through lateral connections, so old tasks are not forgotten. With A3C on Pong variants, Atari and 3D mazes, 2-column nets reach 209% mean transfer on Pong Soup vs 181% for finetuning (author-reported).Paper · 2016-06-05Deep Reinforcement Learning for Dialogue GenerationTunes a seq2seq dialogue model with policy gradients on a hand-set reward: low likelihood of dull replies, new information between turns, and forward/backward coherence. Starts from a mutual-information model, then tunes on simulated two-agent dialogues. Author-reported: 4.48 vs 3.40 simulated turns over the MI baseline.Paper · 2016-04-13Online Multi-Target Tracking Using Recurrent Neural NetworksOnline multi-target tracking with RNNs: an RNN predicts and updates every target's box, an LSTM solves per-frame data association one target at a time, and an existence probability handles births and deaths. Trained on synthetic trajectories. Author-reported on MOTChallenge 2015 test: 19.0% MOTA at 165.2 FPS, with no appearance features.Paper · 2016-04-12Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector MachinesFacial expression recognition from landmark motion in video, using geometry only. Elastic bunch graph tracks 52 landmarks, normalised to the first frame. Multi-class AdaBoost selects features that are scored by DTW distance to per-class median prototypes. An RBF SVM on the boosted features is best. On CK+ (5-fold CV, author-reported): 95.17% (AdaBoost with DTW) and 97.35% (SVM).Paper · 2016-03-12Sequential Short-Text Classification with Recurrent and Convolutional Neural NetworksClassifies each short text in a sequence (dialog utterances) with an LSTM or CNN encoder and a two-layer feedforward head that also sees the vectors of earlier texts. Author-reported best runs: 66.2% (DSTC 4, LSTM), 84.6% (MRDA, CNN) and 73.1% (SwDA, CNN) accuracy. History at the vector level helped more than at the label level.Paper · 2016-03-04High precision grasp pose detection in dense clutterGPD samples 6-DoF parallel-jaw grasp candidates from a point cloud and classifies each with a LeNet-style CNN on 15-channel projected voxel images. Occlusion channels, CAD pretraining and object category knowledge raise recall at 99% precision. Author-reported: 93% grasp success in dense clutter on a Baxter (288 attempts), up from 73% in the authors' prior work.Paper · 2016-01-25A Taxonomy of Deep Convolutional Neural Nets for Computer VisionRecipe-style survey of convolutional neural networks for computer vision. It starts from AlexNet, explains the building blocks (convolution, pooling, ReLU, dropout, SGD) and lists the main variants: region-based, fully convolutional, two-stream video, CNN-RNN captioning and VQA, multi-task and siamese networks. It runs no experiments; its numbers are quoted from cited works.Paper · 2016-01-11Deep Learning over Multi-field Categorical Data: A Case Study on User Response PredictionTwo deep networks for ad click-through prediction on one-hot, multi-field categorical features. FNN puts a factorisation-machine embedding layer under tanh layers. SNN uses a sigmoid bottom layer pre-trained with sampled-negative RBMs or denoising autoencoders. Author-reported: FNN reaches 70.70% AUC on pooled iPinYou data versus 68.81% for logistic regression.Paper · 2015-12-29Feed-Forward Networks with Attention Can Solve Some Long-Term Memory ProblemsFeed-forward attention pools a sequence into one vector: each step gets a scalar score from its own state, and a softmax over time weights the states. On the synthetic addition and multiplication memory tasks, a 100-unit model reaches perfect accuracy for lengths 50 to 10000 (author-reported); an unweighted mean over time fails on multiplication at the longest lengths and on mixed lengths.Paper · 2015-12-17Continuous online sequence learning with an unsupervised neural network modelHTM sequence memory predicts the next element of a stream from sparse binary codes (40 of 2048 columns active) and dendritic segments trained in one pass by a Hebbian-style rule. It keeps several predictions open until context disambiguates them. Author-reported: perfect prediction on artificial sequences up to order 100; NYC taxi error comparable to LSTM, below ARIMA, ELM and ESN.Paper · 2015-10-12Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly BenchmarkNumenta Anomaly Benchmark (NAB): labelled real and synthetic time series with anomaly windows, and a streaming score that rewards early detections and penalises false positives. Scores are normalised so a perfect detector gets 100 and a null detector 0. Author-reported standard-profile scores: Numenta HTM 64.7, Twitter ADVec 47.1, Etsy Skyline 35.7.Paper · 2015-09-30Convolutional Networks on Graphs for Learning Molecular FingerprintsReplaces the fixed hash and indexing steps of circular molecular fingerprints with differentiable ones (degree-specific neighbour sums, softmax pooling per layer), trained end-to-end. Author-reported: with a one-hidden-layer network on top, it matches or beats circular fingerprints on solubility, drug efficacy and photovoltaic efficiency.Paper · 2015-08-31Neural Machine Translation of Rare Words with Subword UnitsSplits rare and unseen words into learned subword units, using byte pair encoding (BPE), adapted from a compression method, so an NMT model needs no back-off dictionary. On WMT 2015 English-German and English-Russian, subword ensembles gain up to 1.1 and 1.3 BLEU over a dictionary baseline (author-reported).Paper · 2015-07-07A Survey and Classification of Controlled Natural LanguagesSurveys 100 English-based controlled natural languages (CNLs) and proposes the PENS scheme, which rates each language in five classes each for precision, expressiveness, naturalness and simplicity. Finds that CNLs form one scattered cloud between English and propositional logic. A survey with no algorithm or experiments; headline result: the 100-language catalogue and 25 observed PENS classes.Paper · 2015-06-01Imaging Time-Series to Improve Classification and ImputationEncodes a 1-D time series as a 2-D image: Gramian angular fields (GASF/GADF) from polar-angle cosines and sines, and Markov transition fields (MTF) from quantile transition statistics. Tiled CNNs on the stacked images give the lowest error on 9 of 20 UCR datasets (author-reported). A denoising autoencoder on GASF images cuts interpolation MSE by 12.18%-48.02% against raw data.Paper · 2015-05-12Improving Computer-aided Detection using Convolutional Neural Networks and Random View AggregationAdds a second stage to existing CADe detectors. Each candidate is cropped into many random 2D or 2.5D views (several scales, random translations and rotations); a ConvNet scores every view and the scores are averaged. Author-reported sensitivity at 3 FPs per patient: 57% to 70% (bone metastases), 43% to 77% (lymph nodes), 58% to 75% (colonic polyps).Paper · 2015-05-03Highway NetworksEach layer blends a plain transform H(x) with its input x through a learned sigmoid gate, so information can pass unchanged across many layers. Negative gate-bias initialisation starts stacks near carry behaviour. Author-reported: 100-layer highway nets train with plain SGD where plain nets degrade, and a 19-layer convolutional highway net reaches 92.24% on CIFAR-10 test.Paper · 2015-02-17On Sex, Evolution, and the Multiplicative Weights Update AlgorithmShows that the allele marginals of sexual population dynamics (recombination before or after selection) follow multiplicative-weights updates for any fitness matrix, recombination rate and population distribution, dropping the weak-selection and product-distribution assumptions of an earlier theory. Author-reported: only diminishing regret follows, which convergence already implies.Paper · 2014-05-30Comparing and Combining Sentiment Analysis MethodsCompares eight sentiment tools (emoticons, LIWC, SentiStrength, SentiWordNet, SenticNet, SASA, Happiness Index, PANAS-t) on coverage and agreement over Twitter events and six human-labeled datasets. A combined method using seven of them reaches 95% coverage, with F 0.730 on the labeled data (author-reported).Paper · 2013-12-31Medical Image Fusion: A survey of the state of the artSurvey of medical image fusion up to 2013. Describes a two-stage pipeline (registration, then feature or decision fusion), groups methods into morphology, knowledge-based, wavelet, neural network, fuzzy logic and other (PCA/ICA, SVM, transforms), and catalogues fusion work by imaging modality (MRI, CT, PET, SPECT, ultrasound) and organ. No experiments of its own.Paper · 2013-12-21Intriguing properties of neural networksShows that deep image classifiers have blind spots: a tiny, optimised input change flips the label (adversarial examples), and these also fool other models and models trained on other data. Argues single hidden units are no more meaningful than random directions. ImageNet evidence is mostly qualitative; quantitative tables use small MNIST models. Author-reported.Paper · 2013-11-07An Efficiently Solvable Quadratic Program for Stabilizing Dynamic LocomotionA whole-body walking controller for a 34-DOF humanoid: a time-varying LQR on the ZMP model gives a cost-to-go that a convex QP minimises under full-robot dynamics, torque, friction and no-slip constraints. A custom active-set solver reuses the previous step's active set and solves one linear system in most steps, reaching about 1 kHz (author-reported).Paper · 2013-05-08Class Imbalance Problem in Data Mining ReviewA short review of the class imbalance problem in classification. It sorts the remedies into three groups: data sampling (random under-sampling, random over-sampling, SMOTE), algorithm changes (one-class, cost-sensitive, SVM-based, boosting) and feature selection. It compares five methods in a qualitative table. It reports no experiments of its own.Paper · 2013-04-11Rotational Projection Statistics for 3D Local Surface Description and Object RecognitionRoPS describes a 3D surface patch by rotating its neighbour points about the axes of a local frame, projecting them onto three planes and keeping four central moments and entropy per histogram (135 numbers). A scatter-matrix frame with sign disambiguation lets one correspondence fix a pose. Author-reported recognition: 100% Bologna, 98.9% UWA, 95.4% Queen's, 96.0% Ca' Foscari.Paper · 2012-11-16Lasso Screening Rules via Dual Polytope ProjectionSafe screening rules (DPP, then EDPP) for the Lasso. Before solving, they drop features whose coefficients are provably zero, using a ball around the dual solution at a nearby penalty; EDPP shrinks that ball with two projection facts. Author-reported: about two orders of magnitude faster on large real matrices along a penalty path.Paper · 2011-12-19Additive Gaussian ProcessesA Gaussian process whose kernel sums products of one-dimensional base kernels over all interaction orders, one variance per order. Newton-Girard identities make the sum O(D^2). Author-reported, 10-fold: lowest regression MSE of five models on concrete and servo; best or tied on most other sets.Paper · 2008-04-09Fast k Nearest Neighbor Search using GPUBrute-force k-nearest-neighbour search on NVIDIA GPUs with CUDA: all distances are computed in parallel and the k smallest are kept per query. Against Matlab, C and kd-tree baselines in Table 1, the authors report up to 120x speed-up over Matlab brute force (author-reported).